2018-11-26 13:25:12 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2018-11-26 13:25:31 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-26 13:25:48 [DEBUG   ] Got command RPT from MCS: ref #633528
2018-11-26 13:25:48 [DEBUG   ] summary = ERROR
2018-11-26 13:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633528  13 58448 48348709 A  ERRORERROR'
2018-11-26 13:25:48 [DEBUG   ] Got command RPT from MCS: ref #633529
2018-11-26 13:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633529  77 58448 48348710 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:25:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:25:50 [WARNING ] (13, 1, 'bot')
2018-11-26 13:25:50 [WARNING ] (13, 1, 'bot')
2018-11-26 13:25:50 [WARNING ] (13, 1, 'bot')
2018-11-26 13:25:50 [WARNING ] (13, 1, 'bot')
2018-11-26 13:25:50 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:25:50 [DEBUG   ] Finished updating board flags in 1.478 seconds
2018-11-26 13:25:51 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-26 13:26:11 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 13:26:32 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 13:26:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:26:50 [WARNING ] (13, 1, 'bot')
2018-11-26 13:26:50 [WARNING ] (13, 1, 'bot')
2018-11-26 13:26:50 [WARNING ] (13, 1, 'bot')
2018-11-26 13:26:50 [WARNING ] (13, 1, 'bot')
2018-11-26 13:26:50 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:26:50 [DEBUG   ] Finished updating board flags in 1.486 seconds
2018-11-26 13:26:52 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 13:27:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:27:32 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:27:41 [DEBUG   ] Got command RPT from MCS: ref #633545
2018-11-26 13:27:41 [DEBUG   ] summary = ERROR
2018-11-26 13:27:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633545  13 58448 48461701 A  ERRORERROR'
2018-11-26 13:27:41 [DEBUG   ] Got command RPT from MCS: ref #633546
2018-11-26 13:27:41 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:27:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633546  77 58448 48461905 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:27:42 [DEBUG   ] Got command RPT from MCS: ref #633547
2018-11-26 13:27:42 [DEBUG   ] summary = ERROR
2018-11-26 13:27:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633547  13 58448 48462109 A  ERRORERROR'
2018-11-26 13:27:42 [DEBUG   ] Got command RPT from MCS: ref #633548
2018-11-26 13:27:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:27:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633548  77 58448 48462313 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:27:42 [DEBUG   ] Got command RPT from MCS: ref #633549
2018-11-26 13:27:42 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 13:27:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633549  12 58448 48462505 A  ERRORB–™š'
2018-11-26 13:27:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:27:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:27:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:27:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:27:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:27:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:27:51 [DEBUG   ] Finished updating board flags in 2.196 seconds
2018-11-26 13:27:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:28:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:28:32 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:28:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:28:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:28:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:28:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:28:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:28:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:28:51 [DEBUG   ] Finished updating board flags in 1.985 seconds
2018-11-26 13:28:52 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:29:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:29:32 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:29:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:29:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:29:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:29:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:29:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:29:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:29:51 [DEBUG   ] Finished updating board flags in 1.897 seconds
2018-11-26 13:29:52 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:30:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:30:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:30:48 [DEBUG   ] Got command RPT from MCS: ref #633554
2018-11-26 13:30:48 [DEBUG   ] summary = ERROR
2018-11-26 13:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633554  13 58448 48648705 A  ERRORERROR'
2018-11-26 13:30:48 [DEBUG   ] Got command RPT from MCS: ref #633555
2018-11-26 13:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633555  77 58448 48648706 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:30:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:30:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:30:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:30:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:30:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:30:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:30:51 [DEBUG   ] Finished updating board flags in 1.724 seconds
2018-11-26 13:30:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:31:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:31:32 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:31:50 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 13:31:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:31:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:31:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:31:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:31:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:31:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:31:51 [DEBUG   ] Finished updating board flags in 1.640 seconds
2018-11-26 13:31:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:32:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:32:32 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:32:41 [DEBUG   ] Got command RPT from MCS: ref #633577
2018-11-26 13:32:41 [DEBUG   ] summary = ERROR
2018-11-26 13:32:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633577  13 58448 48761789 A  ERRORERROR'
2018-11-26 13:32:41 [DEBUG   ] Got command RPT from MCS: ref #633578
2018-11-26 13:32:41 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:32:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633578  77 58448 48761993 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:32:42 [DEBUG   ] Got command RPT from MCS: ref #633579
2018-11-26 13:32:42 [DEBUG   ] summary = ERROR
2018-11-26 13:32:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633579  13 58448 48762185 A  ERRORERROR'
2018-11-26 13:32:42 [DEBUG   ] Got command RPT from MCS: ref #633581
2018-11-26 13:32:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:32:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633581  77 58448 48762389 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:32:42 [DEBUG   ] Got command RPT from MCS: ref #633582
2018-11-26 13:32:42 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 13:32:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633582  12 58448 48762593 A  ERRORB–™š'
2018-11-26 13:32:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 13:32:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-26 13:32:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:32:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 13:32:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:32:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:32:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:32:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:32:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:32:51 [DEBUG   ] Finished updating board flags in 1.546 seconds
2018-11-26 13:32:52 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:33:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:33:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:33:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 13:33:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:33:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:33:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:33:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:33:51 [WARNING ] (13, 1, 'bot')
2018-11-26 13:33:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:33:51 [DEBUG   ] Finished updating board flags in 1.555 seconds
2018-11-26 13:33:52 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:34:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:34:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:34:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:34:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 13:34:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:34:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:34:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:34:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:34:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:34:52 [DEBUG   ] Finished updating board flags in 1.617 seconds
2018-11-26 13:34:52 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:35:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:35:32 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:35:48 [DEBUG   ] Got command RPT from MCS: ref #633591
2018-11-26 13:35:48 [DEBUG   ] summary = ERROR
2018-11-26 13:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633591  13 58448 48948713 A  ERRORERROR'
2018-11-26 13:35:48 [DEBUG   ] Got command RPT from MCS: ref #633592
2018-11-26 13:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633592  77 58448 48948714 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:35:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:35:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:35:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:35:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:35:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:35:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:35:52 [DEBUG   ] Finished updating board flags in 1.637 seconds
2018-11-26 13:35:52 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:36:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:36:32 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:36:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:36:52 [DEBUG   ] Finished updating ant stats in 1.257 seconds
2018-11-26 13:36:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:36:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:36:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:36:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:36:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:36:52 [DEBUG   ] Finished updating board flags in 2.341 seconds
2018-11-26 13:37:12 [DEBUG   ] Finished updating ant stats in 1.145 seconds
2018-11-26 13:37:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 13:37:41 [DEBUG   ] Got command RPT from MCS: ref #633612
2018-11-26 13:37:41 [DEBUG   ] summary = ERROR
2018-11-26 13:37:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633612  13 58448 49061865 A  ERRORERROR'
2018-11-26 13:37:42 [DEBUG   ] Got command RPT from MCS: ref #633613
2018-11-26 13:37:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:37:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633613  77 58448 49062069 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:37:42 [DEBUG   ] Got command RPT from MCS: ref #633614
2018-11-26 13:37:42 [DEBUG   ] summary = ERROR
2018-11-26 13:37:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633614  13 58448 49062273 A  ERRORERROR'
2018-11-26 13:37:42 [DEBUG   ] Got command RPT from MCS: ref #633615
2018-11-26 13:37:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:37:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633615  77 58448 49062478 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:37:42 [DEBUG   ] Got command RPT from MCS: ref #633616
2018-11-26 13:37:42 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 13:37:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633616  12 58448 49062669 A  ERRORB—™š'
2018-11-26 13:37:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:37:52 [DEBUG   ] Finished updating ant stats in 1.047 seconds
2018-11-26 13:37:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:37:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:37:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:37:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:37:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:37:52 [DEBUG   ] Finished updating board flags in 2.092 seconds
2018-11-26 13:38:12 [DEBUG   ] Finished updating ant stats in 0.958 seconds
2018-11-26 13:38:32 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 13:38:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:38:52 [DEBUG   ] Finished updating ant stats in 0.847 seconds
2018-11-26 13:38:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:38:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:38:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:38:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:38:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:38:52 [DEBUG   ] Finished updating board flags in 1.998 seconds
2018-11-26 13:39:12 [DEBUG   ] Finished updating ant stats in 0.760 seconds
2018-11-26 13:39:32 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 13:39:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:39:52 [DEBUG   ] Finished updating ant stats in 0.645 seconds
2018-11-26 13:39:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:39:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:39:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:39:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:39:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:39:52 [DEBUG   ] Finished updating board flags in 1.814 seconds
2018-11-26 13:40:12 [DEBUG   ] Finished updating ant stats in 0.558 seconds
2018-11-26 13:40:32 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 13:40:48 [DEBUG   ] Got command RPT from MCS: ref #633621
2018-11-26 13:40:48 [DEBUG   ] summary = ERROR
2018-11-26 13:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633621  13 58448 49248713 A  ERRORERROR'
2018-11-26 13:40:48 [DEBUG   ] Got command RPT from MCS: ref #633622
2018-11-26 13:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633622  77 58448 49248714 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:40:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:40:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 13:40:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:40:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:40:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:40:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:40:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:40:52 [DEBUG   ] Finished updating board flags in 1.783 seconds
2018-11-26 13:40:53 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 13:41:13 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 13:41:33 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:41:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:41:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:41:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:41:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:41:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:41:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:41:52 [DEBUG   ] Finished updating board flags in 1.597 seconds
2018-11-26 13:41:53 [DEBUG   ] Finished updating ant stats in 1.193 seconds
2018-11-26 13:42:13 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 13:42:33 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 13:42:41 [DEBUG   ] Got command RPT from MCS: ref #633640
2018-11-26 13:42:41 [DEBUG   ] summary = ERROR
2018-11-26 13:42:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633640  13 58448 49361945 A  ERRORERROR'
2018-11-26 13:42:42 [DEBUG   ] Got command RPT from MCS: ref #633641
2018-11-26 13:42:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:42:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633641  77 58448 49362149 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:42:42 [DEBUG   ] Got command RPT from MCS: ref #633642
2018-11-26 13:42:42 [DEBUG   ] summary = ERROR
2018-11-26 13:42:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633642  13 58448 49362353 A  ERRORERROR'
2018-11-26 13:42:42 [DEBUG   ] Got command RPT from MCS: ref #633644
2018-11-26 13:42:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:42:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633644  77 58448 49362545 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:42:42 [DEBUG   ] Got command RPT from MCS: ref #633645
2018-11-26 13:42:42 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 13:42:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633645  12 58448 49362749 A  ERRORB–™š'
2018-11-26 13:42:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:42:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:42:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:42:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:42:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:42:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:42:52 [DEBUG   ] Finished updating board flags in 1.407 seconds
2018-11-26 13:42:53 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 13:43:13 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 13:43:33 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 13:43:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:43:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:43:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:43:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:43:52 [WARNING ] (13, 1, 'bot')
2018-11-26 13:43:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:43:52 [DEBUG   ] Finished updating board flags in 1.394 seconds
2018-11-26 13:43:53 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 13:44:13 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 13:44:33 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 13:44:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:44:53 [DEBUG   ] Finished updating ant stats in 1.046 seconds
2018-11-26 13:44:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:44:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:44:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:44:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:44:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:44:53 [DEBUG   ] Finished updating board flags in 2.232 seconds
2018-11-26 13:45:13 [DEBUG   ] Finished updating ant stats in 0.958 seconds
2018-11-26 13:45:33 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 13:45:48 [DEBUG   ] Got command RPT from MCS: ref #633654
2018-11-26 13:45:48 [DEBUG   ] summary = ERROR
2018-11-26 13:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633654  13 58448 49548709 A  ERRORERROR'
2018-11-26 13:45:48 [DEBUG   ] Got command RPT from MCS: ref #633655
2018-11-26 13:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633655  77 58448 49548710 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:45:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:45:53 [DEBUG   ] Finished updating ant stats in 0.845 seconds
2018-11-26 13:45:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:45:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:45:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:45:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:45:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:45:53 [DEBUG   ] Finished updating board flags in 2.192 seconds
2018-11-26 13:46:13 [DEBUG   ] Finished updating ant stats in 0.757 seconds
2018-11-26 13:46:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 13:46:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:46:53 [DEBUG   ] Finished updating ant stats in 0.647 seconds
2018-11-26 13:46:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 13:46:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:46:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:46:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:46:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:46:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:46:53 [DEBUG   ] Finished updating board flags in 2.001 seconds
2018-11-26 13:47:13 [DEBUG   ] Finished updating ant stats in 0.557 seconds
2018-11-26 13:47:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-26 13:47:42 [DEBUG   ] Got command RPT from MCS: ref #633675
2018-11-26 13:47:42 [DEBUG   ] summary = ERROR
2018-11-26 13:47:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633675  13 58448 49662033 A  ERRORERROR'
2018-11-26 13:47:42 [DEBUG   ] Got command RPT from MCS: ref #633676
2018-11-26 13:47:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:47:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633676  77 58448 49662225 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:47:42 [DEBUG   ] Got command RPT from MCS: ref #633677
2018-11-26 13:47:42 [DEBUG   ] summary = ERROR
2018-11-26 13:47:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633677  13 58448 49662429 A  ERRORERROR'
2018-11-26 13:47:42 [DEBUG   ] Got command RPT from MCS: ref #633678
2018-11-26 13:47:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:47:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633678  77 58448 49662633 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:47:42 [DEBUG   ] Got command RPT from MCS: ref #633679
2018-11-26 13:47:42 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 13:47:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633679  12 58448 49662825 A  ERRORB•™š'
2018-11-26 13:47:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:47:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 13:47:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:47:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:47:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:47:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:47:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:47:53 [DEBUG   ] Finished updating board flags in 1.987 seconds
2018-11-26 13:47:54 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-26 13:48:14 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 13:48:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:48:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:48:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:48:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:48:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:48:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:48:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:48:53 [DEBUG   ] Finished updating board flags in 1.819 seconds
2018-11-26 13:48:54 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:49:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:49:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:49:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:49:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:49:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:49:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:49:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:49:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:49:53 [DEBUG   ] Finished updating board flags in 1.839 seconds
2018-11-26 13:49:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:50:14 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:50:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:50:48 [DEBUG   ] Got command RPT from MCS: ref #633684
2018-11-26 13:50:48 [DEBUG   ] summary = ERROR
2018-11-26 13:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633684  13 58448 49848713 A  ERRORERROR'
2018-11-26 13:50:48 [DEBUG   ] Got command RPT from MCS: ref #633685
2018-11-26 13:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633685  77 58448 49848714 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:50:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:50:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:50:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:50:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:50:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:50:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:50:53 [DEBUG   ] Finished updating board flags in 1.834 seconds
2018-11-26 13:50:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:51:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 13:51:34 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 13:51:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:51:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:51:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:51:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:51:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:51:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:51:53 [DEBUG   ] Finished updating board flags in 1.698 seconds
2018-11-26 13:51:54 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 13:52:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 13:52:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 13:52:42 [DEBUG   ] Got command RPT from MCS: ref #633707
2018-11-26 13:52:42 [DEBUG   ] summary = ERROR
2018-11-26 13:52:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633707  13 58448 49962105 A  ERRORERROR'
2018-11-26 13:52:42 [DEBUG   ] Got command RPT from MCS: ref #633708
2018-11-26 13:52:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:52:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633708  77 58448 49962309 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:52:42 [DEBUG   ] Got command RPT from MCS: ref #633709
2018-11-26 13:52:42 [DEBUG   ] summary = ERROR
2018-11-26 13:52:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633709  13 58448 49962513 A  ERRORERROR'
2018-11-26 13:52:42 [DEBUG   ] Got command RPT from MCS: ref #633711
2018-11-26 13:52:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:52:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633711  77 58448 49962705 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:52:42 [DEBUG   ] Got command RPT from MCS: ref #633712
2018-11-26 13:52:42 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 13:52:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633712  12 58448 49962909 A  ERRORB•™š'
2018-11-26 13:52:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:52:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:52:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:52:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:52:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:52:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:52:53 [DEBUG   ] Finished updating board flags in 1.624 seconds
2018-11-26 13:52:54 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 13:53:14 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 13:53:34 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 13:53:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 13:53:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:53:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:53:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:53:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:53:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:53:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:53:53 [DEBUG   ] Finished updating board flags in 1.587 seconds
2018-11-26 13:53:54 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 13:54:14 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 13:54:34 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 13:54:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:54:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:54:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:54:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:54:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:54:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:54:53 [DEBUG   ] Finished updating board flags in 1.558 seconds
2018-11-26 13:54:54 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2018-11-26 13:55:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 13:55:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 13:55:48 [DEBUG   ] Got command RPT from MCS: ref #633721
2018-11-26 13:55:48 [DEBUG   ] summary = ERROR
2018-11-26 13:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633721  13 58448 50148717 A  ERRORERROR'
2018-11-26 13:55:48 [DEBUG   ] Got command RPT from MCS: ref #633722
2018-11-26 13:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633722  77 58448 50148718 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:55:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:55:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:55:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:55:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:55:53 [WARNING ] (13, 1, 'bot')
2018-11-26 13:55:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:55:53 [DEBUG   ] Finished updating board flags in 1.548 seconds
2018-11-26 13:55:54 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 13:56:14 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 13:56:34 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 13:56:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 13:56:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:56:54 [DEBUG   ] Finished updating ant stats in 0.647 seconds
2018-11-26 13:56:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:56:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:56:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:56:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:56:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:56:54 [DEBUG   ] Finished updating board flags in 2.124 seconds
2018-11-26 13:57:14 [DEBUG   ] Finished updating ant stats in 0.558 seconds
2018-11-26 13:57:35 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 13:57:42 [DEBUG   ] Got command RPT from MCS: ref #633738
2018-11-26 13:57:42 [DEBUG   ] summary = ERROR
2018-11-26 13:57:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633738  13 58448 50262189 A  ERRORERROR'
2018-11-26 13:57:42 [DEBUG   ] Got command RPT from MCS: ref #633739
2018-11-26 13:57:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:57:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633739  77 58448 50262393 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:57:42 [DEBUG   ] Got command RPT from MCS: ref #633740
2018-11-26 13:57:42 [DEBUG   ] summary = ERROR
2018-11-26 13:57:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633740  13 58448 50262586 A  ERRORERROR'
2018-11-26 13:57:42 [DEBUG   ] Got command RPT from MCS: ref #633741
2018-11-26 13:57:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 13:57:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633741  77 58448 50262790 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 13:57:42 [DEBUG   ] Got command RPT from MCS: ref #633742
2018-11-26 13:57:42 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 13:57:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633742  12 58448 50262993 A  ERRORB–™š'
2018-11-26 13:57:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:57:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:57:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:57:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:57:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:57:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:57:54 [DEBUG   ] Finished updating board flags in 1.968 seconds
2018-11-26 13:57:55 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 13:58:00 [DEBUG   ] Got command TBN from MCS: ref #633747
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '1', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '2', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] tbn = exited with status True, code 0x00
2018-11-26 13:58:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBN   633747   8 58448 50280458 A  ERROR'
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '3', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '4', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '5', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '6', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '7', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '8', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '9', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '10', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '11', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '12', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '13', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '14', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '15', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '16', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '17', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '18', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '19', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '20', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '21', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '22', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '23', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '24', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '25', '1543240681']
2018-11-26 13:58:00 [DEBUG   ] spawn_process: at 2018-11-26 13:58:00 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '26', '1543240681']
2018-11-26 13:58:00 [INFO    ] Finished spawning TBN in 0.053 s
2018-11-26 13:58:15 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:58:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:58:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:58:55 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 13:58:55 [WARNING ] (13, 1, 'bot')
2018-11-26 13:58:55 [WARNING ] (13, 1, 'bot')
2018-11-26 13:58:55 [WARNING ] (13, 1, 'bot')
2018-11-26 13:58:55 [WARNING ] (13, 1, 'bot')
2018-11-26 13:58:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:58:55 [DEBUG   ] Finished updating board flags in 2.861 seconds
2018-11-26 13:59:15 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 13:59:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 13:59:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 13:59:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:59:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:59:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:59:54 [WARNING ] (13, 1, 'bot')
2018-11-26 13:59:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 13:59:54 [DEBUG   ] Finished updating board flags in 1.790 seconds
2018-11-26 13:59:55 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:00:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:00:35 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:00:48 [DEBUG   ] Got command RPT from MCS: ref #633748
2018-11-26 14:00:48 [DEBUG   ] summary = ERROR
2018-11-26 14:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633748  13 58448 50448710 A  ERRORERROR'
2018-11-26 14:00:48 [DEBUG   ] Got command RPT from MCS: ref #633749
2018-11-26 14:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633749  77 58448 50448710 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:00:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:00:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:00:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:00:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:00:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:00:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:00:54 [DEBUG   ] Finished updating board flags in 1.652 seconds
2018-11-26 14:00:55 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:01:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:01:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:01:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:01:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:01:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:01:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:01:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:01:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:01:54 [DEBUG   ] Finished updating board flags in 1.535 seconds
2018-11-26 14:01:55 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:02:11 [DEBUG   ] Got command TBW from MCS: ref #633770
2018-11-26 14:02:11 [DEBUG   ] tbw = exited with status True, code 0x00
2018-11-26 14:02:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBW   633770   8 58448 50531534 A  ERROR'
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 pkill tbn
2018-11-26 14:02:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 pkill tbn
2018-11-26 14:02:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 pkill tbn
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa27', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '1', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa24', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '2', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa23', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '3', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa18', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '4', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa10', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '5', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa32', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '6', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa11', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '7', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa30', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '8', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa12', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '9', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa8', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '10', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa13', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '11', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa26', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '12', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa14', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '13', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa22', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '14', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa20', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '15', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa1', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '16', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa5', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '17', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa25', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '18', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa35', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '19', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa19', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '20', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa21', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '21', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa6', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '22', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa7', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '23', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa33', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '24', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa15', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '25', '12000000']
2018-11-26 14:02:15 [DEBUG   ] spawn_process: at 2018-11-26 14:02:15 running ['/usr/local/bin/lwa28', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '26', '12000000']
2018-11-26 14:02:15 [INFO    ] Finished spawning TBW
2018-11-26 14:02:42 [DEBUG   ] Got command RPT from MCS: ref #633775
2018-11-26 14:02:42 [DEBUG   ] summary = ERROR
2018-11-26 14:02:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633775  13 58448 50562261 A  ERRORERROR'
2018-11-26 14:02:42 [DEBUG   ] Got command RPT from MCS: ref #633776
2018-11-26 14:02:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:02:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633776  77 58448 50562465 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:02:42 [DEBUG   ] Got command RPT from MCS: ref #633777
2018-11-26 14:02:42 [DEBUG   ] summary = ERROR
2018-11-26 14:02:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633777  13 58448 50562669 A  ERRORERROR'
2018-11-26 14:02:42 [DEBUG   ] Got command RPT from MCS: ref #633779
2018-11-26 14:02:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:02:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633779  77 58448 50562861 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:02:43 [DEBUG   ] Got command RPT from MCS: ref #633781
2018-11-26 14:02:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:02:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633781  12 58448 50563065 A  ERRORB–™š'
2018-11-26 14:02:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:02:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:02:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:02:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:02:54 [WARNING ] (13, 1, 'bot')
2018-11-26 14:02:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:02:54 [DEBUG   ] Finished updating board flags in 1.167 seconds
2018-11-26 14:03:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:03:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:03:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:03:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:03:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:03:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:03:55 [DEBUG   ] Finished updating board flags in 2.137 seconds
2018-11-26 14:04:03 [INFO    ] __waitForTBW - TBW finished in 108.770 s
2018-11-26 14:04:04 [DEBUG   ] Finished updating ant stats in 0.430 seconds
2018-11-26 14:04:04 [INFO    ] Started antennas statistics background thread
2018-11-26 14:04:04 [INFO    ] Resuming TBN at 38.100 MHz after TBW acquisition
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '1', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '2', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '3', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '4', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '5', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '6', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '7', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '8', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '9', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '10', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '11', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '12', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '13', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '14', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '15', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '16', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '17', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '18', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '19', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '20', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '21', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '22', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '23', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '24', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '25', '1543241046']
2018-11-26 14:04:04 [DEBUG   ] spawn_process: at 2018-11-26 14:04:04 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '26', '1543241046']
2018-11-26 14:04:04 [INFO    ] Finished spawning TBN in 0.052 s
2018-11-26 14:04:24 [DEBUG   ] Finished updating ant stats in 0.469 seconds
2018-11-26 14:04:44 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-26 14:04:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:04:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:04:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:04:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:04:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:04:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:04:55 [DEBUG   ] Finished updating board flags in 2.178 seconds
2018-11-26 14:05:04 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 14:05:24 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 14:05:44 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-26 14:05:48 [DEBUG   ] Got command RPT from MCS: ref #633794
2018-11-26 14:05:48 [DEBUG   ] summary = ERROR
2018-11-26 14:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633794  13 58448 50748717 A  ERRORERROR'
2018-11-26 14:05:48 [DEBUG   ] Got command RPT from MCS: ref #633795
2018-11-26 14:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633795  77 58448 50748718 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:05:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:05:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:05:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:05:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:05:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:05:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:05:55 [DEBUG   ] Finished updating board flags in 1.858 seconds
2018-11-26 14:06:05 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-26 14:06:25 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 14:06:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:06:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:06:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:06:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:06:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:06:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:06:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:06:55 [DEBUG   ] Finished updating board flags in 1.817 seconds
2018-11-26 14:07:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:07:25 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 14:07:42 [DEBUG   ] Got command RPT from MCS: ref #633815
2018-11-26 14:07:42 [DEBUG   ] summary = ERROR
2018-11-26 14:07:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633815  13 58448 50862345 A  ERRORERROR'
2018-11-26 14:07:42 [DEBUG   ] Got command RPT from MCS: ref #633816
2018-11-26 14:07:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:07:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633816  77 58448 50862549 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:07:42 [DEBUG   ] Got command RPT from MCS: ref #633817
2018-11-26 14:07:42 [DEBUG   ] summary = ERROR
2018-11-26 14:07:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633817  13 58448 50862753 A  ERRORERROR'
2018-11-26 14:07:42 [DEBUG   ] Got command RPT from MCS: ref #633818
2018-11-26 14:07:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:07:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633818  77 58448 50862954 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:07:43 [DEBUG   ] Got command RPT from MCS: ref #633819
2018-11-26 14:07:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:07:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633819  12 58448 50863145 A  ERRORB—™š'
2018-11-26 14:07:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:07:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:07:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:07:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:07:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:07:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:07:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:07:55 [DEBUG   ] Finished updating board flags in 1.782 seconds
2018-11-26 14:08:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:08:25 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:08:45 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 14:08:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:08:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:08:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:08:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:08:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:08:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:08:55 [DEBUG   ] Finished updating board flags in 1.625 seconds
2018-11-26 14:09:05 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:09:25 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:09:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:09:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:09:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:09:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:09:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:09:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:09:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:09:55 [DEBUG   ] Finished updating board flags in 1.574 seconds
2018-11-26 14:10:05 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:10:25 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:10:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:10:48 [DEBUG   ] Got command RPT from MCS: ref #633824
2018-11-26 14:10:48 [DEBUG   ] summary = ERROR
2018-11-26 14:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633824  13 58448 51048721 A  ERRORERROR'
2018-11-26 14:10:48 [DEBUG   ] Got command RPT from MCS: ref #633825
2018-11-26 14:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633825  77 58448 51048722 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:10:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:10:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2018-11-26 14:10:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:10:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:10:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:10:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:10:55 [WARNING ] (25, 5, 'bot')
2018-11-26 14:10:55 [WARNING ] (25, 5, 'bot')
2018-11-26 14:10:55 [WARNING ] (25, 5, 'bot')
2018-11-26 14:10:55 [WARNING ] (25, 5, 'bot')
2018-11-26 14:10:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:10:55 [DEBUG   ] Finished updating board flags in 1.527 seconds
2018-11-26 14:11:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:11:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:11:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:11:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:11:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:11:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:11:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:11:55 [WARNING ] (13, 1, 'bot')
2018-11-26 14:11:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:11:55 [DEBUG   ] Finished updating board flags in 1.578 seconds
2018-11-26 14:12:05 [DEBUG   ] Finished updating ant stats in 1.194 seconds
2018-11-26 14:12:25 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 14:12:42 [DEBUG   ] Got command RPT from MCS: ref #633843
2018-11-26 14:12:42 [DEBUG   ] summary = ERROR
2018-11-26 14:12:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633843  13 58448 51162429 A  ERRORERROR'
2018-11-26 14:12:42 [DEBUG   ] Got command RPT from MCS: ref #633844
2018-11-26 14:12:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:12:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633844  77 58448 51162621 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:12:42 [DEBUG   ] Got command RPT from MCS: ref #633845
2018-11-26 14:12:42 [DEBUG   ] summary = ERROR
2018-11-26 14:12:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633845  13 58448 51162839 A  ERRORERROR'
2018-11-26 14:12:43 [DEBUG   ] Got command RPT from MCS: ref #633847
2018-11-26 14:12:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:12:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633847  77 58448 51163029 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:12:43 [DEBUG   ] Got command RPT from MCS: ref #633848
2018-11-26 14:12:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:12:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633848  12 58448 51163233 A  ERRORB—™š'
2018-11-26 14:12:45 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 14:12:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:12:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 14:12:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:12:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:12:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:12:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:12:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:12:56 [DEBUG   ] Finished updating board flags in 2.218 seconds
2018-11-26 14:13:05 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 14:13:25 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 14:13:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 14:13:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:13:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:13:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:13:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:13:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:13:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:13:56 [DEBUG   ] Finished updating board flags in 2.088 seconds
2018-11-26 14:14:05 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 14:14:25 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 14:14:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 14:14:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:14:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:14:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:14:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:14:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:14:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:14:56 [DEBUG   ] Finished updating board flags in 2.030 seconds
2018-11-26 14:15:05 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 14:15:25 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 14:15:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 14:15:48 [DEBUG   ] Got command RPT from MCS: ref #633857
2018-11-26 14:15:48 [DEBUG   ] summary = ERROR
2018-11-26 14:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633857  13 58448 51348721 A  ERRORERROR'
2018-11-26 14:15:48 [DEBUG   ] Got command RPT from MCS: ref #633858
2018-11-26 14:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633858  77 58448 51348722 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:15:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:15:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:15:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:15:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:15:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:15:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:15:56 [DEBUG   ] Finished updating board flags in 2.020 seconds
2018-11-26 14:16:05 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 14:16:25 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 14:16:45 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 14:16:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:16:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:16:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:16:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:16:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:16:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:16:56 [DEBUG   ] Finished updating board flags in 1.979 seconds
2018-11-26 14:17:05 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 14:17:25 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 14:17:42 [DEBUG   ] Got command RPT from MCS: ref #633884
2018-11-26 14:17:42 [DEBUG   ] summary = ERROR
2018-11-26 14:17:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633884  13 58448 51462509 A  ERRORERROR'
2018-11-26 14:17:42 [DEBUG   ] Got command RPT from MCS: ref #633885
2018-11-26 14:17:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:17:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633885  77 58448 51462719 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:17:42 [DEBUG   ] Got command RPT from MCS: ref #633886
2018-11-26 14:17:42 [DEBUG   ] summary = ERROR
2018-11-26 14:17:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633886  13 58448 51462909 A  ERRORERROR'
2018-11-26 14:17:43 [DEBUG   ] Got command RPT from MCS: ref #633887
2018-11-26 14:17:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:17:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633887  77 58448 51463113 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:17:43 [DEBUG   ] Got command RPT from MCS: ref #633888
2018-11-26 14:17:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:17:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633888  12 58448 51463305 A  ERRORB–™š'
2018-11-26 14:17:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 14:17:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 14:17:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:17:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:17:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:17:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:17:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:17:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:17:56 [DEBUG   ] Finished updating board flags in 1.902 seconds
2018-11-26 14:18:05 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 14:18:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:18:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:18:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:18:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 14:18:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:18:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:18:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:18:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:18:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:18:56 [DEBUG   ] Finished updating board flags in 1.690 seconds
2018-11-26 14:19:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:19:25 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 14:19:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:19:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-26 14:19:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:19:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 14:19:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:19:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:19:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:19:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:19:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:19:56 [DEBUG   ] Finished updating board flags in 1.634 seconds
2018-11-26 14:20:05 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:20:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:20:45 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 14:20:48 [DEBUG   ] Got command RPT from MCS: ref #633898
2018-11-26 14:20:48 [DEBUG   ] summary = ERROR
2018-11-26 14:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633898  13 58448 51648713 A  ERRORERROR'
2018-11-26 14:20:48 [DEBUG   ] Got command RPT from MCS: ref #633899
2018-11-26 14:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633899  77 58448 51648714 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:20:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:20:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:20:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:20:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:20:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:20:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:20:56 [DEBUG   ] Finished updating board flags in 1.564 seconds
2018-11-26 14:21:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:21:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:21:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:21:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:21:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:21:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:21:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:21:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:21:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:21:56 [DEBUG   ] Finished updating board flags in 1.416 seconds
2018-11-26 14:22:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 14:22:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:22:42 [DEBUG   ] Got command RPT from MCS: ref #633921
2018-11-26 14:22:42 [DEBUG   ] summary = ERROR
2018-11-26 14:22:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633921  13 58448 51762589 A  ERRORERROR'
2018-11-26 14:22:42 [DEBUG   ] Got command RPT from MCS: ref #633922
2018-11-26 14:22:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:22:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633922  77 58448 51762793 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:22:42 [DEBUG   ] Got command RPT from MCS: ref #633923
2018-11-26 14:22:42 [DEBUG   ] summary = ERROR
2018-11-26 14:22:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633923  13 58448 51762985 A  ERRORERROR'
2018-11-26 14:22:43 [DEBUG   ] Got command RPT from MCS: ref #633925
2018-11-26 14:22:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:22:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633925  77 58448 51763189 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:22:43 [DEBUG   ] Got command RPT from MCS: ref #633926
2018-11-26 14:22:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:22:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633926  12 58448 51763393 A  ERRORB—™š'
2018-11-26 14:22:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:22:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:22:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:22:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:22:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:22:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:22:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:22:56 [DEBUG   ] Finished updating board flags in 1.499 seconds
2018-11-26 14:23:05 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:23:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 14:23:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:23:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:23:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:23:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:23:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:23:56 [WARNING ] (13, 1, 'bot')
2018-11-26 14:23:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:23:56 [DEBUG   ] Finished updating board flags in 1.558 seconds
2018-11-26 14:24:05 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:24:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:24:45 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 14:24:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:24:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:24:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:24:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:24:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:24:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:24:57 [DEBUG   ] Finished updating board flags in 2.254 seconds
2018-11-26 14:25:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:25:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:25:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:25:48 [DEBUG   ] Got command RPT from MCS: ref #633935
2018-11-26 14:25:48 [DEBUG   ] summary = ERROR
2018-11-26 14:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633935  13 58448 51948721 A  ERRORERROR'
2018-11-26 14:25:48 [DEBUG   ] Got command RPT from MCS: ref #633936
2018-11-26 14:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633936  77 58448 51948722 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:25:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:25:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:25:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:25:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:25:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:25:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:25:57 [DEBUG   ] Finished updating board flags in 2.099 seconds
2018-11-26 14:26:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 14:26:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:26:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:26:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:26:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:26:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:26:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:26:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:26:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:26:57 [DEBUG   ] Finished updating board flags in 2.021 seconds
2018-11-26 14:27:05 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:27:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 14:27:42 [DEBUG   ] Got command RPT from MCS: ref #633952
2018-11-26 14:27:42 [DEBUG   ] summary = ERROR
2018-11-26 14:27:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633952  13 58448 52062669 A  ERRORERROR'
2018-11-26 14:27:42 [DEBUG   ] Got command RPT from MCS: ref #633953
2018-11-26 14:27:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:27:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633953  77 58448 52062873 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:27:43 [DEBUG   ] Got command RPT from MCS: ref #633954
2018-11-26 14:27:43 [DEBUG   ] summary = ERROR
2018-11-26 14:27:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633954  13 58448 52063065 A  ERRORERROR'
2018-11-26 14:27:43 [DEBUG   ] Got command RPT from MCS: ref #633955
2018-11-26 14:27:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:27:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633955  77 58448 52063269 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:27:43 [DEBUG   ] Got command RPT from MCS: ref #633956
2018-11-26 14:27:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:27:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633956  12 58448 52063473 A  ERRORB–™š'
2018-11-26 14:27:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:27:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:27:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:27:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:27:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:27:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:27:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:27:57 [DEBUG   ] Finished updating board flags in 1.966 seconds
2018-11-26 14:28:05 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:28:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:28:45 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 14:28:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:28:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:28:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:28:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:28:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:28:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:28:57 [DEBUG   ] Finished updating board flags in 1.810 seconds
2018-11-26 14:29:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:29:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:29:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:29:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 14:29:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:29:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:29:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:29:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:29:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:29:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:29:57 [DEBUG   ] Finished updating board flags in 1.918 seconds
2018-11-26 14:30:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 14:30:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:30:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:30:48 [DEBUG   ] Got command RPT from MCS: ref #633961
2018-11-26 14:30:48 [DEBUG   ] summary = ERROR
2018-11-26 14:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633961  13 58448 52248721 A  ERRORERROR'
2018-11-26 14:30:48 [DEBUG   ] Got command RPT from MCS: ref #633962
2018-11-26 14:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633962  77 58448 52248722 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:30:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:30:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:30:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:30:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:30:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:30:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:30:57 [DEBUG   ] Finished updating board flags in 1.685 seconds
2018-11-26 14:31:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:31:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 14:31:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:31:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:31:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:31:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:31:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:31:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:31:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:31:57 [DEBUG   ] Finished updating board flags in 1.611 seconds
2018-11-26 14:32:05 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:32:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:32:42 [DEBUG   ] Got command RPT from MCS: ref #633984
2018-11-26 14:32:42 [DEBUG   ] summary = ERROR
2018-11-26 14:32:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633984  13 58448 52362749 A  ERRORERROR'
2018-11-26 14:32:42 [DEBUG   ] Got command RPT from MCS: ref #633985
2018-11-26 14:32:42 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:32:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633985  77 58448 52362953 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:32:43 [DEBUG   ] Got command RPT from MCS: ref #633986
2018-11-26 14:32:43 [DEBUG   ] summary = ERROR
2018-11-26 14:32:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633986  13 58448 52363157 A  ERRORERROR'
2018-11-26 14:32:43 [DEBUG   ] Got command RPT from MCS: ref #633988
2018-11-26 14:32:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:32:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633988  77 58448 52363349 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:32:43 [DEBUG   ] Got command RPT from MCS: ref #633989
2018-11-26 14:32:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:32:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633989  12 58448 52363553 A  ERRORB–™š'
2018-11-26 14:32:45 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 14:32:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:32:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:32:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:32:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:32:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:32:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:32:57 [DEBUG   ] Finished updating board flags in 1.584 seconds
2018-11-26 14:33:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:33:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:33:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:33:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:33:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:33:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:33:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:33:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:33:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:33:57 [DEBUG   ] Finished updating board flags in 1.421 seconds
2018-11-26 14:34:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 14:34:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:34:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:34:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:34:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:34:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:34:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:34:57 [WARNING ] (13, 1, 'bot')
2018-11-26 14:34:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:34:57 [DEBUG   ] Finished updating board flags in 1.558 seconds
2018-11-26 14:35:05 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 14:35:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 14:35:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 14:35:48 [DEBUG   ] Got command RPT from MCS: ref #633998
2018-11-26 14:35:48 [DEBUG   ] summary = ERROR
2018-11-26 14:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633998  13 58448 52548721 A  ERRORERROR'
2018-11-26 14:35:48 [DEBUG   ] Got command RPT from MCS: ref #633999
2018-11-26 14:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   633999  77 58448 52548722 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:35:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:35:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:35:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:35:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:35:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:35:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:35:58 [DEBUG   ] Finished updating board flags in 1.581 seconds
2018-11-26 14:36:05 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2018-11-26 14:36:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:36:45 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 14:36:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:36:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:36:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:36:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:36:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:36:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:36:58 [DEBUG   ] Finished updating board flags in 2.163 seconds
2018-11-26 14:37:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 14:37:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:37:42 [DEBUG   ] Got command RPT from MCS: ref #634019
2018-11-26 14:37:42 [DEBUG   ] summary = ERROR
2018-11-26 14:37:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634019  13 58448 52662831 A  ERRORERROR'
2018-11-26 14:37:43 [DEBUG   ] Got command RPT from MCS: ref #634020
2018-11-26 14:37:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:37:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634020  77 58448 52663025 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:37:43 [DEBUG   ] Got command RPT from MCS: ref #634021
2018-11-26 14:37:43 [DEBUG   ] summary = ERROR
2018-11-26 14:37:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634021  13 58448 52663229 A  ERRORERROR'
2018-11-26 14:37:43 [DEBUG   ] Got command RPT from MCS: ref #634022
2018-11-26 14:37:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:37:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634022  77 58448 52663433 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:37:43 [DEBUG   ] Got command RPT from MCS: ref #634023
2018-11-26 14:37:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:37:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634023  12 58448 52663637 A  ERRORB—™š'
2018-11-26 14:37:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 14:37:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:37:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:37:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:37:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:37:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:37:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:37:58 [DEBUG   ] Finished updating board flags in 2.013 seconds
2018-11-26 14:38:05 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 14:38:26 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2018-11-26 14:38:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:38:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:38:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:38:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:38:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:38:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:38:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:38:58 [DEBUG   ] Finished updating board flags in 1.892 seconds
2018-11-26 14:39:05 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:39:25 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 14:39:46 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2018-11-26 14:39:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:39:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:39:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:39:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:39:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:39:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:39:58 [DEBUG   ] Finished updating board flags in 1.806 seconds
2018-11-26 14:40:05 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:40:26 [DEBUG   ] Finished updating ant stats in 1.604 seconds
2018-11-26 14:40:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 14:40:48 [DEBUG   ] Got command RPT from MCS: ref #634028
2018-11-26 14:40:48 [DEBUG   ] summary = ERROR
2018-11-26 14:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634028  13 58448 52848725 A  ERRORERROR'
2018-11-26 14:40:48 [DEBUG   ] Got command RPT from MCS: ref #634029
2018-11-26 14:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634029  77 58448 52848726 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:40:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:40:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:40:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:40:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:40:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:40:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:40:58 [DEBUG   ] Finished updating board flags in 1.821 seconds
2018-11-26 14:41:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 14:41:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:41:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:41:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:41:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:41:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:41:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:41:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:41:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:41:58 [DEBUG   ] Finished updating board flags in 1.807 seconds
2018-11-26 14:42:05 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 14:42:26 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2018-11-26 14:42:42 [DEBUG   ] Got command RPT from MCS: ref #634047
2018-11-26 14:42:42 [DEBUG   ] summary = ERROR
2018-11-26 14:42:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634047  13 58448 52962913 A  ERRORERROR'
2018-11-26 14:42:43 [DEBUG   ] Got command RPT from MCS: ref #634048
2018-11-26 14:42:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:42:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634048  77 58448 52963117 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:42:43 [DEBUG   ] Got command RPT from MCS: ref #634049
2018-11-26 14:42:43 [DEBUG   ] summary = ERROR
2018-11-26 14:42:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634049  13 58448 52963309 A  ERRORERROR'
2018-11-26 14:42:43 [DEBUG   ] Got command RPT from MCS: ref #634051
2018-11-26 14:42:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:42:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634051  77 58448 52963513 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:42:43 [DEBUG   ] Got command RPT from MCS: ref #634052
2018-11-26 14:42:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:42:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634052  12 58448 52963717 A  ERRORB–™š'
2018-11-26 14:42:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:42:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:42:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:42:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:42:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:42:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:42:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:42:58 [DEBUG   ] Finished updating board flags in 1.895 seconds
2018-11-26 14:43:05 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:43:25 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 14:43:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 14:43:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 14:43:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:43:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:43:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:43:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:43:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:43:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:43:58 [DEBUG   ] Finished updating board flags in 1.812 seconds
2018-11-26 14:44:05 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:44:26 [DEBUG   ] Finished updating ant stats in 1.603 seconds
2018-11-26 14:44:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 14:44:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 14:44:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:44:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:44:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:44:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:44:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:44:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:44:58 [DEBUG   ] Finished updating board flags in 1.657 seconds
2018-11-26 14:45:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 14:45:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:45:46 [DEBUG   ] Finished updating ant stats in 1.603 seconds
2018-11-26 14:45:48 [DEBUG   ] Got command RPT from MCS: ref #634061
2018-11-26 14:45:48 [DEBUG   ] summary = ERROR
2018-11-26 14:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634061  13 58448 53148725 A  ERRORERROR'
2018-11-26 14:45:48 [DEBUG   ] Got command RPT from MCS: ref #634062
2018-11-26 14:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634062  77 58448 53148726 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:45:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:45:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:45:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:45:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:45:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:45:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:45:58 [DEBUG   ] Finished updating board flags in 1.619 seconds
2018-11-26 14:46:05 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 14:46:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 14:46:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 14:46:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:46:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:46:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:46:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:46:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:46:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:46:58 [DEBUG   ] Finished updating board flags in 1.599 seconds
2018-11-26 14:47:05 [DEBUG   ] Finished updating ant stats in 0.593 seconds
2018-11-26 14:47:26 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 14:47:42 [DEBUG   ] Got command RPT from MCS: ref #634082
2018-11-26 14:47:42 [DEBUG   ] summary = ERROR
2018-11-26 14:47:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634082  13 58448 53262989 A  ERRORERROR'
2018-11-26 14:47:43 [DEBUG   ] Got command RPT from MCS: ref #634083
2018-11-26 14:47:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:47:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634083  77 58448 53263193 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:47:43 [DEBUG   ] Got command RPT from MCS: ref #634084
2018-11-26 14:47:43 [DEBUG   ] summary = ERROR
2018-11-26 14:47:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634084  13 58448 53263397 A  ERRORERROR'
2018-11-26 14:47:43 [DEBUG   ] Got command RPT from MCS: ref #634085
2018-11-26 14:47:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:47:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634085  77 58448 53263601 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:47:43 [DEBUG   ] Got command RPT from MCS: ref #634086
2018-11-26 14:47:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:47:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634086  12 58448 53263793 A  ERRORB—™š'
2018-11-26 14:47:46 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 14:47:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:47:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:47:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:47:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:47:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:47:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:47:58 [DEBUG   ] Finished updating board flags in 1.390 seconds
2018-11-26 14:48:06 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:48:26 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:48:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:48:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:48:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:48:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:48:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:48:58 [WARNING ] (13, 1, 'bot')
2018-11-26 14:48:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:48:58 [DEBUG   ] Finished updating board flags in 1.444 seconds
2018-11-26 14:49:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:49:26 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:49:46 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:49:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:49:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:49:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:49:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:49:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:49:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:49:59 [DEBUG   ] Finished updating board flags in 2.181 seconds
2018-11-26 14:50:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:50:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:50:46 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:50:48 [DEBUG   ] Got command RPT from MCS: ref #634091
2018-11-26 14:50:48 [DEBUG   ] summary = ERROR
2018-11-26 14:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634091  13 58448 53448725 A  ERRORERROR'
2018-11-26 14:50:48 [DEBUG   ] Got command RPT from MCS: ref #634092
2018-11-26 14:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634092  77 58448 53448726 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:50:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:50:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:50:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:50:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:50:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:50:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:50:59 [DEBUG   ] Finished updating board flags in 2.074 seconds
2018-11-26 14:51:06 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:51:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:51:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:51:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:51:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:51:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:51:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:51:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:51:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:51:59 [DEBUG   ] Finished updating board flags in 1.908 seconds
2018-11-26 14:52:06 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:52:26 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:52:43 [DEBUG   ] Got command RPT from MCS: ref #634114
2018-11-26 14:52:43 [DEBUG   ] summary = ERROR
2018-11-26 14:52:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634114  13 58448 53563077 A  ERRORERROR'
2018-11-26 14:52:43 [DEBUG   ] Got command RPT from MCS: ref #634115
2018-11-26 14:52:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:52:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634115  77 58448 53563269 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:52:43 [DEBUG   ] Got command RPT from MCS: ref #634116
2018-11-26 14:52:43 [DEBUG   ] summary = ERROR
2018-11-26 14:52:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634116  13 58448 53563473 A  ERRORERROR'
2018-11-26 14:52:43 [DEBUG   ] Got command RPT from MCS: ref #634118
2018-11-26 14:52:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:52:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634118  77 58448 53563677 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:52:43 [DEBUG   ] Got command RPT from MCS: ref #634120
2018-11-26 14:52:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:52:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634120  12 58448 53563869 A  ERRORB–™š'
2018-11-26 14:52:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:52:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:52:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:52:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:52:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:52:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:52:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:52:59 [DEBUG   ] Finished updating board flags in 1.867 seconds
2018-11-26 14:53:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:53:26 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:53:46 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:53:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:53:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 14:53:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:53:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:53:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:53:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:53:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:53:59 [DEBUG   ] Finished updating board flags in 1.749 seconds
2018-11-26 14:54:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:54:26 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:54:46 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:54:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:54:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:54:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:54:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:54:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:54:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:54:59 [DEBUG   ] Finished updating board flags in 1.622 seconds
2018-11-26 14:55:06 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:55:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:55:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:55:48 [DEBUG   ] Got command RPT from MCS: ref #634128
2018-11-26 14:55:48 [DEBUG   ] summary = ERROR
2018-11-26 14:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634128  13 58448 53748721 A  ERRORERROR'
2018-11-26 14:55:48 [DEBUG   ] Got command RPT from MCS: ref #634129
2018-11-26 14:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634129  77 58448 53748722 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:55:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:55:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:55:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:55:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:55:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:55:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:55:59 [DEBUG   ] Finished updating board flags in 1.562 seconds
2018-11-26 14:56:06 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 14:56:26 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:56:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:56:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:56:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:56:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:56:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:56:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:56:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:56:59 [DEBUG   ] Finished updating board flags in 1.562 seconds
2018-11-26 14:57:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:57:26 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:57:43 [DEBUG   ] Got command RPT from MCS: ref #634145
2018-11-26 14:57:43 [DEBUG   ] summary = ERROR
2018-11-26 14:57:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634145  13 58448 53863153 A  ERRORERROR'
2018-11-26 14:57:43 [DEBUG   ] Got command RPT from MCS: ref #634146
2018-11-26 14:57:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:57:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634146  77 58448 53863357 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:57:43 [DEBUG   ] Got command RPT from MCS: ref #634147
2018-11-26 14:57:43 [DEBUG   ] summary = ERROR
2018-11-26 14:57:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634147  13 58448 53863549 A  ERRORERROR'
2018-11-26 14:57:43 [DEBUG   ] Got command RPT from MCS: ref #634148
2018-11-26 14:57:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 14:57:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634148  77 58448 53863753 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 14:57:43 [DEBUG   ] Got command RPT from MCS: ref #634149
2018-11-26 14:57:43 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 14:57:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634149  12 58448 53863957 A  ERRORB•™š'
2018-11-26 14:57:46 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:57:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:57:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:57:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:57:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:57:59 [WARNING ] (13, 1, 'bot')
2018-11-26 14:57:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:57:59 [DEBUG   ] Finished updating board flags in 1.589 seconds
2018-11-26 14:58:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:58:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:58:46 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:58:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 14:59:00 [WARNING ] (13, 1, 'bot')
2018-11-26 14:59:00 [WARNING ] (13, 1, 'bot')
2018-11-26 14:59:00 [WARNING ] (13, 1, 'bot')
2018-11-26 14:59:00 [WARNING ] (13, 1, 'bot')
2018-11-26 14:59:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 14:59:00 [DEBUG   ] Finished updating board flags in 2.180 seconds
2018-11-26 14:59:06 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 14:59:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:59:46 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 14:59:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:00:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:00:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:00:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:00:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:00:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:00:00 [DEBUG   ] Finished updating board flags in 1.874 seconds
2018-11-26 15:00:06 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 15:00:26 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 15:00:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:00:48 [DEBUG   ] Got command RPT from MCS: ref #634154
2018-11-26 15:00:48 [DEBUG   ] summary = ERROR
2018-11-26 15:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634154  13 58448 54048721 A  ERRORERROR'
2018-11-26 15:00:48 [DEBUG   ] Got command RPT from MCS: ref #634155
2018-11-26 15:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634155  77 58448 54048722 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:00:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:01:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:01:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:01:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:01:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:01:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:01:00 [DEBUG   ] Finished updating board flags in 1.824 seconds
2018-11-26 15:01:06 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 15:01:26 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 15:01:46 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 15:01:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:02:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:02:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:02:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:02:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:02:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:02:00 [DEBUG   ] Finished updating board flags in 1.787 seconds
2018-11-26 15:02:06 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:02:26 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:02:43 [DEBUG   ] Got command RPT from MCS: ref #634177
2018-11-26 15:02:43 [DEBUG   ] summary = ERROR
2018-11-26 15:02:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634177  13 58448 54163229 A  ERRORERROR'
2018-11-26 15:02:43 [DEBUG   ] Got command RPT from MCS: ref #634178
2018-11-26 15:02:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:02:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634178  77 58448 54163433 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:02:43 [DEBUG   ] Got command RPT from MCS: ref #634179
2018-11-26 15:02:43 [DEBUG   ] summary = ERROR
2018-11-26 15:02:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634179  13 58448 54163637 A  ERRORERROR'
2018-11-26 15:02:43 [DEBUG   ] Got command RPT from MCS: ref #634181
2018-11-26 15:02:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:02:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634181  77 58448 54163841 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:02:44 [DEBUG   ] Got command RPT from MCS: ref #634182
2018-11-26 15:02:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:02:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634182  12 58448 54164033 A  ERRORB—™š'
2018-11-26 15:02:46 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 15:02:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 15:02:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:03:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:03:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:03:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:03:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:03:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:03:00 [DEBUG   ] Finished updating board flags in 1.613 seconds
2018-11-26 15:03:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 15:03:26 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:03:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:03:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:04:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:04:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:04:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:04:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:04:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:04:00 [DEBUG   ] Finished updating board flags in 1.612 seconds
2018-11-26 15:04:06 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 15:04:26 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 15:04:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:04:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:05:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:05:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:05:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:05:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:05:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:05:00 [DEBUG   ] Finished updating board flags in 1.711 seconds
2018-11-26 15:05:06 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 15:05:26 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 15:05:46 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 15:05:48 [DEBUG   ] Got command RPT from MCS: ref #634191
2018-11-26 15:05:48 [DEBUG   ] summary = ERROR
2018-11-26 15:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634191  13 58448 54348729 A  ERRORERROR'
2018-11-26 15:05:48 [DEBUG   ] Got command RPT from MCS: ref #634192
2018-11-26 15:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634192  77 58448 54348730 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:05:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:06:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:06:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:06:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:06:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:06:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:06:00 [DEBUG   ] Finished updating board flags in 1.474 seconds
2018-11-26 15:06:06 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 15:06:26 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:06:46 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:06:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:07:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:07:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:07:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:07:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:07:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:07:00 [DEBUG   ] Finished updating board flags in 1.479 seconds
2018-11-26 15:07:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:07:26 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:07:43 [DEBUG   ] Got command RPT from MCS: ref #634212
2018-11-26 15:07:43 [DEBUG   ] summary = ERROR
2018-11-26 15:07:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634212  13 58448 54463321 A  ERRORERROR'
2018-11-26 15:07:43 [DEBUG   ] Got command RPT from MCS: ref #634213
2018-11-26 15:07:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:07:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634213  77 58448 54463513 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:07:43 [DEBUG   ] Got command RPT from MCS: ref #634214
2018-11-26 15:07:43 [DEBUG   ] summary = ERROR
2018-11-26 15:07:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634214  13 58448 54463717 A  ERRORERROR'
2018-11-26 15:07:43 [DEBUG   ] Got command RPT from MCS: ref #634215
2018-11-26 15:07:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:07:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634215  77 58448 54463921 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:07:44 [DEBUG   ] Got command RPT from MCS: ref #634216
2018-11-26 15:07:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:07:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634216  12 58448 54464113 A  ERRORB—™š'
2018-11-26 15:07:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:08:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:08:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:08:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:08:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:08:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:08:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:08:00 [DEBUG   ] Finished updating board flags in 1.483 seconds
2018-11-26 15:08:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:08:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:08:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:09:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:09:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 15:09:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:09:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:09:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:09:00 [WARNING ] (13, 1, 'bot')
2018-11-26 15:09:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:09:00 [DEBUG   ] Finished updating board flags in 1.485 seconds
2018-11-26 15:09:06 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:09:26 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:09:46 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:10:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:10:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 15:10:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:10:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:10:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:10:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:10:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:10:01 [DEBUG   ] Finished updating board flags in 2.195 seconds
2018-11-26 15:10:06 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:10:26 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:10:46 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:10:48 [DEBUG   ] Got command RPT from MCS: ref #634221
2018-11-26 15:10:48 [DEBUG   ] summary = ERROR
2018-11-26 15:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634221  13 58448 54648729 A  ERRORERROR'
2018-11-26 15:10:48 [DEBUG   ] Got command RPT from MCS: ref #634222
2018-11-26 15:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634222  77 58448 54648730 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:11:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:11:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:11:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:11:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:11:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:11:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:11:01 [DEBUG   ] Finished updating board flags in 2.092 seconds
2018-11-26 15:11:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:11:26 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 15:11:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:12:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:12:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:12:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:12:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:12:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:12:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:12:01 [DEBUG   ] Finished updating board flags in 2.035 seconds
2018-11-26 15:12:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:12:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:12:43 [DEBUG   ] Got command RPT from MCS: ref #634240
2018-11-26 15:12:43 [DEBUG   ] summary = ERROR
2018-11-26 15:12:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634240  13 58448 54763397 A  ERRORERROR'
2018-11-26 15:12:43 [DEBUG   ] Got command RPT from MCS: ref #634241
2018-11-26 15:12:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:12:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634241  77 58448 54763601 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:12:43 [DEBUG   ] Got command RPT from MCS: ref #634242
2018-11-26 15:12:43 [DEBUG   ] summary = ERROR
2018-11-26 15:12:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634242  13 58448 54763793 A  ERRORERROR'
2018-11-26 15:12:43 [DEBUG   ] Got command RPT from MCS: ref #634244
2018-11-26 15:12:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:12:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634244  77 58448 54763997 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:12:44 [DEBUG   ] Got command RPT from MCS: ref #634245
2018-11-26 15:12:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:12:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634245  12 58448 54764201 A  ERRORB–™š'
2018-11-26 15:12:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:13:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:13:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:13:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:13:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:13:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:13:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:13:01 [DEBUG   ] Finished updating board flags in 1.856 seconds
2018-11-26 15:13:06 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:13:26 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:13:46 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 15:14:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:14:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:14:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:14:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:14:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:14:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:14:01 [DEBUG   ] Finished updating board flags in 1.820 seconds
2018-11-26 15:14:06 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:14:26 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:14:46 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:15:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa16, chip 1
2018-11-26 15:15:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 15:15:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:15:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:15:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:15:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:15:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:15:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:15:01 [DEBUG   ] Finished updating board flags in 1.795 seconds
2018-11-26 15:15:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:15:26 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:15:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:15:48 [DEBUG   ] Got command RPT from MCS: ref #634254
2018-11-26 15:15:48 [DEBUG   ] summary = ERROR
2018-11-26 15:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634254  13 58448 54948725 A  ERRORERROR'
2018-11-26 15:15:48 [DEBUG   ] Got command RPT from MCS: ref #634255
2018-11-26 15:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634255  77 58448 54948726 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:16:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:16:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:16:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:16:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:16:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:16:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:16:01 [DEBUG   ] Finished updating board flags in 1.604 seconds
2018-11-26 15:16:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:16:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:16:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:17:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:17:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:17:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:17:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:17:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:17:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:17:01 [DEBUG   ] Finished updating board flags in 1.763 seconds
2018-11-26 15:17:06 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:17:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:17:43 [DEBUG   ] Got command RPT from MCS: ref #634275
2018-11-26 15:17:43 [DEBUG   ] summary = ERROR
2018-11-26 15:17:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634275  13 58448 55063473 A  ERRORERROR'
2018-11-26 15:17:43 [DEBUG   ] Got command RPT from MCS: ref #634276
2018-11-26 15:17:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:17:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634276  77 58448 55063677 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:17:43 [DEBUG   ] Got command RPT from MCS: ref #634277
2018-11-26 15:17:43 [DEBUG   ] summary = ERROR
2018-11-26 15:17:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634277  13 58448 55063881 A  ERRORERROR'
2018-11-26 15:17:44 [DEBUG   ] Got command RPT from MCS: ref #634278
2018-11-26 15:17:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:17:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634278  77 58448 55064073 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:17:44 [DEBUG   ] Got command RPT from MCS: ref #634279
2018-11-26 15:17:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:17:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634279  12 58448 55064277 A  ERRORB—™š'
2018-11-26 15:17:46 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:18:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:18:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 15:18:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:18:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:18:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:18:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:18:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:18:01 [DEBUG   ] Finished updating board flags in 1.453 seconds
2018-11-26 15:18:06 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:18:26 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:18:46 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:19:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:19:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:19:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:19:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:19:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:19:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:19:01 [DEBUG   ] Finished updating board flags in 1.429 seconds
2018-11-26 15:19:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:19:26 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:19:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:20:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 15:20:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:20:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:20:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:20:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:20:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:20:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:20:01 [DEBUG   ] Finished updating board flags in 1.428 seconds
2018-11-26 15:20:06 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 15:20:26 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 15:20:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 15:20:48 [DEBUG   ] Got command RPT from MCS: ref #634284
2018-11-26 15:20:48 [DEBUG   ] summary = ERROR
2018-11-26 15:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634284  13 58448 55248729 A  ERRORERROR'
2018-11-26 15:20:48 [DEBUG   ] Got command RPT from MCS: ref #634285
2018-11-26 15:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634285  77 58448 55248730 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:21:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:21:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 15:21:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:21:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:21:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:21:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:21:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:21:01 [DEBUG   ] Finished updating board flags in 1.437 seconds
2018-11-26 15:21:06 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 15:21:26 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 15:21:46 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 15:22:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:22:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:22:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:22:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:22:01 [WARNING ] (13, 1, 'bot')
2018-11-26 15:22:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:22:01 [DEBUG   ] Finished updating board flags in 1.449 seconds
2018-11-26 15:22:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 15:22:26 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 15:22:43 [DEBUG   ] Got command RPT from MCS: ref #634307
2018-11-26 15:22:43 [DEBUG   ] summary = ERROR
2018-11-26 15:22:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634307  13 58448 55363557 A  ERRORERROR'
2018-11-26 15:22:43 [DEBUG   ] Got command RPT from MCS: ref #634308
2018-11-26 15:22:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:22:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634308  77 58448 55363749 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:22:43 [DEBUG   ] Got command RPT from MCS: ref #634309
2018-11-26 15:22:43 [DEBUG   ] summary = ERROR
2018-11-26 15:22:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634309  13 58448 55363953 A  ERRORERROR'
2018-11-26 15:22:44 [DEBUG   ] Got command RPT from MCS: ref #634311
2018-11-26 15:22:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:22:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634311  77 58448 55364157 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:22:44 [DEBUG   ] Got command RPT from MCS: ref #634312
2018-11-26 15:22:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:22:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634312  12 58448 55364361 A  ERRORB—™š'
2018-11-26 15:22:46 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 15:23:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:23:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:23:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:23:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:23:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:23:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:23:02 [DEBUG   ] Finished updating board flags in 1.592 seconds
2018-11-26 15:23:06 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 15:23:26 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-26 15:23:46 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-26 15:24:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 15:24:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:24:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:24:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:24:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:24:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:24:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:24:02 [DEBUG   ] Finished updating board flags in 2.164 seconds
2018-11-26 15:24:07 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2018-11-26 15:24:27 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 15:24:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:25:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:25:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:25:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:25:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:25:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:25:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:25:02 [DEBUG   ] Finished updating board flags in 1.891 seconds
2018-11-26 15:25:07 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:25:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:25:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:25:48 [DEBUG   ] Got command RPT from MCS: ref #634321
2018-11-26 15:25:48 [DEBUG   ] summary = ERROR
2018-11-26 15:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634321  13 58448 55548725 A  ERRORERROR'
2018-11-26 15:25:48 [DEBUG   ] Got command RPT from MCS: ref #634322
2018-11-26 15:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634322  77 58448 55548726 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:26:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:26:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:26:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:26:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:26:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:26:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:26:02 [DEBUG   ] Finished updating board flags in 1.815 seconds
2018-11-26 15:26:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:26:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:26:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:27:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:27:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:27:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:27:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:27:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:27:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:27:02 [DEBUG   ] Finished updating board flags in 1.781 seconds
2018-11-26 15:27:07 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2018-11-26 15:27:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:27:43 [DEBUG   ] Got command RPT from MCS: ref #634338
2018-11-26 15:27:43 [DEBUG   ] summary = ERROR
2018-11-26 15:27:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634338  13 58448 55663637 A  ERRORERROR'
2018-11-26 15:27:43 [DEBUG   ] Got command RPT from MCS: ref #634339
2018-11-26 15:27:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:27:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634339  77 58448 55663829 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:27:44 [DEBUG   ] Got command RPT from MCS: ref #634340
2018-11-26 15:27:44 [DEBUG   ] summary = ERROR
2018-11-26 15:27:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634340  13 58448 55664033 A  ERRORERROR'
2018-11-26 15:27:44 [DEBUG   ] Got command RPT from MCS: ref #634341
2018-11-26 15:27:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:27:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634341  77 58448 55664237 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:27:44 [DEBUG   ] Got command RPT from MCS: ref #634342
2018-11-26 15:27:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:27:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634342  12 58448 55664441 A  ERRORB–™š'
2018-11-26 15:27:47 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 15:28:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:28:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:28:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:28:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:28:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:28:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:28:02 [DEBUG   ] Finished updating board flags in 1.618 seconds
2018-11-26 15:28:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:28:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:28:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:29:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:29:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:29:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:29:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:29:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:29:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:29:02 [DEBUG   ] Finished updating board flags in 1.604 seconds
2018-11-26 15:29:07 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 15:29:27 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:29:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:30:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:30:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:30:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:30:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:30:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:30:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:30:02 [DEBUG   ] Finished updating board flags in 1.595 seconds
2018-11-26 15:30:07 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:30:27 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:30:47 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:30:48 [DEBUG   ] Got command RPT from MCS: ref #634347
2018-11-26 15:30:48 [DEBUG   ] summary = ERROR
2018-11-26 15:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634347  13 58448 55848729 A  ERRORERROR'
2018-11-26 15:30:48 [DEBUG   ] Got command RPT from MCS: ref #634348
2018-11-26 15:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634348  77 58448 55848730 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:31:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:31:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 15:31:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:31:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:31:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:31:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:31:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:31:02 [DEBUG   ] Finished updating board flags in 1.498 seconds
2018-11-26 15:31:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:31:27 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:31:47 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:32:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:32:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:32:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:32:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:32:02 [WARNING ] (13, 1, 'bot')
2018-11-26 15:32:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:32:02 [DEBUG   ] Finished updating board flags in 1.556 seconds
2018-11-26 15:32:07 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:32:27 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:32:43 [DEBUG   ] Got command RPT from MCS: ref #634370
2018-11-26 15:32:43 [DEBUG   ] summary = ERROR
2018-11-26 15:32:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634370  13 58448 55963713 A  ERRORERROR'
2018-11-26 15:32:43 [DEBUG   ] Got command RPT from MCS: ref #634371
2018-11-26 15:32:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:32:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634371  77 58448 55963917 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:32:44 [DEBUG   ] Got command RPT from MCS: ref #634372
2018-11-26 15:32:44 [DEBUG   ] summary = ERROR
2018-11-26 15:32:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634372  13 58448 55964109 A  ERRORERROR'
2018-11-26 15:32:44 [DEBUG   ] Got command RPT from MCS: ref #634374
2018-11-26 15:32:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:32:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634374  77 58448 55964313 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:32:44 [DEBUG   ] Got command RPT from MCS: ref #634375
2018-11-26 15:32:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:32:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634375  12 58448 55964517 A  ERRORB•™š'
2018-11-26 15:32:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:33:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa16, chip 1
2018-11-26 15:33:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:33:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:33:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:33:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:33:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:33:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:33:03 [DEBUG   ] Finished updating board flags in 2.257 seconds
2018-11-26 15:33:07 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 15:33:27 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:33:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:34:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 15:34:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:34:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:34:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:34:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:34:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:34:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:34:03 [DEBUG   ] Finished updating board flags in 2.107 seconds
2018-11-26 15:34:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:34:27 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:34:47 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:35:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:35:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:35:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:35:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:35:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:35:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:35:03 [DEBUG   ] Finished updating board flags in 2.007 seconds
2018-11-26 15:35:07 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2018-11-26 15:35:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 15:35:47 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 15:35:48 [DEBUG   ] Got command RPT from MCS: ref #634384
2018-11-26 15:35:48 [DEBUG   ] summary = ERROR
2018-11-26 15:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634384  13 58448 56148729 A  ERRORERROR'
2018-11-26 15:35:48 [DEBUG   ] Got command RPT from MCS: ref #634385
2018-11-26 15:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634385  77 58448 56148730 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:36:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa16, chip 1
2018-11-26 15:36:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:36:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:36:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:36:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:36:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:36:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:36:03 [DEBUG   ] Finished updating board flags in 2.015 seconds
2018-11-26 15:36:07 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 15:36:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 15:36:48 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-26 15:37:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:37:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:37:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:37:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:37:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:37:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:37:03 [DEBUG   ] Finished updating board flags in 1.967 seconds
2018-11-26 15:37:08 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 15:37:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:37:43 [DEBUG   ] Got command RPT from MCS: ref #634405
2018-11-26 15:37:43 [DEBUG   ] summary = ERROR
2018-11-26 15:37:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634405  13 58448 56263797 A  ERRORERROR'
2018-11-26 15:37:43 [DEBUG   ] Got command RPT from MCS: ref #634406
2018-11-26 15:37:43 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:37:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634406  77 58448 56263989 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:37:44 [DEBUG   ] Got command RPT from MCS: ref #634407
2018-11-26 15:37:44 [DEBUG   ] summary = ERROR
2018-11-26 15:37:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634407  13 58448 56264193 A  ERRORERROR'
2018-11-26 15:37:44 [DEBUG   ] Got command RPT from MCS: ref #634408
2018-11-26 15:37:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:37:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634408  77 58448 56264397 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:37:44 [DEBUG   ] Got command RPT from MCS: ref #634409
2018-11-26 15:37:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:37:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634409  12 58448 56264601 A  ERRORB—™š'
2018-11-26 15:37:48 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:38:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:38:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:38:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:38:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:38:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:38:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:38:03 [DEBUG   ] Finished updating board flags in 1.644 seconds
2018-11-26 15:38:08 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 15:38:28 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:38:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:39:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:39:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:39:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:39:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:39:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:39:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:39:03 [DEBUG   ] Finished updating board flags in 1.643 seconds
2018-11-26 15:39:08 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:39:28 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:39:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:40:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:40:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:40:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:40:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:40:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:40:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:40:03 [DEBUG   ] Finished updating board flags in 1.558 seconds
2018-11-26 15:40:08 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2018-11-26 15:40:28 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 15:40:48 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 15:40:48 [DEBUG   ] Got command RPT from MCS: ref #634414
2018-11-26 15:40:48 [DEBUG   ] summary = ERROR
2018-11-26 15:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634414  13 58448 56448729 A  ERRORERROR'
2018-11-26 15:40:48 [DEBUG   ] Got command RPT from MCS: ref #634415
2018-11-26 15:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634415  77 58448 56448730 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:41:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:41:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:41:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:41:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:41:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:41:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:41:03 [DEBUG   ] Finished updating board flags in 1.412 seconds
2018-11-26 15:41:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 15:41:28 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:41:48 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 15:42:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:42:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:42:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:42:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:42:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:42:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:42:03 [DEBUG   ] Finished updating board flags in 1.475 seconds
2018-11-26 15:42:08 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 15:42:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:42:43 [DEBUG   ] Got command RPT from MCS: ref #634433
2018-11-26 15:42:43 [DEBUG   ] summary = ERROR
2018-11-26 15:42:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634433  13 58448 56563865 A  ERRORERROR'
2018-11-26 15:42:44 [DEBUG   ] Got command RPT from MCS: ref #634434
2018-11-26 15:42:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:42:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634434  77 58448 56564069 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:42:44 [DEBUG   ] Got command RPT from MCS: ref #634435
2018-11-26 15:42:44 [DEBUG   ] summary = ERROR
2018-11-26 15:42:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634435  13 58448 56564273 A  ERRORERROR'
2018-11-26 15:42:44 [DEBUG   ] Got command RPT from MCS: ref #634436
2018-11-26 15:42:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:42:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634436  77 58448 56564477 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:42:44 [DEBUG   ] Got command RPT from MCS: ref #634438
2018-11-26 15:42:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:42:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634438  12 58448 56564669 A  ERRORB–™š'
2018-11-26 15:42:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:43:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:43:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:43:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:43:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:43:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:43:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:43:03 [DEBUG   ] Finished updating board flags in 1.476 seconds
2018-11-26 15:43:08 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:43:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:43:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 15:44:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:44:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:44:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:44:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:44:03 [WARNING ] (13, 1, 'bot')
2018-11-26 15:44:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:44:03 [DEBUG   ] Finished updating board flags in 1.554 seconds
2018-11-26 15:44:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:44:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 15:44:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 15:45:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:45:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 15:45:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:45:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:45:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:45:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:45:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:45:04 [DEBUG   ] Finished updating board flags in 2.197 seconds
2018-11-26 15:45:08 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 15:45:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 15:45:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 15:45:48 [DEBUG   ] Got command RPT from MCS: ref #634447
2018-11-26 15:45:48 [DEBUG   ] summary = ERROR
2018-11-26 15:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634447  13 58448 56748734 A  ERRORERROR'
2018-11-26 15:45:48 [DEBUG   ] Got command RPT from MCS: ref #634448
2018-11-26 15:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634448  77 58448 56748734 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:46:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:46:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 15:46:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:46:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:46:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:46:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:46:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:46:04 [DEBUG   ] Finished updating board flags in 1.900 seconds
2018-11-26 15:46:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 15:46:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 15:46:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 15:47:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 15:47:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:47:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:47:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:47:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:47:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:47:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:47:04 [DEBUG   ] Finished updating board flags in 1.813 seconds
2018-11-26 15:47:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 15:47:28 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 15:47:43 [DEBUG   ] Got command RPT from MCS: ref #634468
2018-11-26 15:47:43 [DEBUG   ] summary = ERROR
2018-11-26 15:47:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634468  13 58448 56863949 A  ERRORERROR'
2018-11-26 15:47:44 [DEBUG   ] Got command RPT from MCS: ref #634469
2018-11-26 15:47:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:47:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634469  77 58448 56864153 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:47:44 [DEBUG   ] Got command RPT from MCS: ref #634470
2018-11-26 15:47:44 [DEBUG   ] summary = ERROR
2018-11-26 15:47:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634470  13 58448 56864345 A  ERRORERROR'
2018-11-26 15:47:44 [DEBUG   ] Got command RPT from MCS: ref #634471
2018-11-26 15:47:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:47:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634471  77 58448 56864549 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:47:44 [DEBUG   ] Got command RPT from MCS: ref #634472
2018-11-26 15:47:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:47:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634472  12 58448 56864753 A  ERRORB—™š'
2018-11-26 15:47:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 15:48:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:48:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:48:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:48:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:48:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:48:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:48:04 [DEBUG   ] Finished updating board flags in 1.821 seconds
2018-11-26 15:48:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 15:48:28 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2018-11-26 15:48:49 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-26 15:49:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:49:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:49:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:49:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:49:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:49:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:49:04 [DEBUG   ] Finished updating board flags in 1.788 seconds
2018-11-26 15:49:09 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 15:49:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:49:49 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:50:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:50:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:50:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:50:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:50:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:50:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:50:04 [DEBUG   ] Finished updating board flags in 1.590 seconds
2018-11-26 15:50:09 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:50:29 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:50:48 [DEBUG   ] Got command RPT from MCS: ref #634477
2018-11-26 15:50:48 [DEBUG   ] summary = ERROR
2018-11-26 15:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634477  13 58448 57048737 A  ERRORERROR'
2018-11-26 15:50:48 [DEBUG   ] Got command RPT from MCS: ref #634478
2018-11-26 15:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634478  77 58448 57048738 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:50:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:51:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 15:51:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:51:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:51:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:51:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:51:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:51:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:51:04 [DEBUG   ] Finished updating board flags in 1.485 seconds
2018-11-26 15:51:09 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:51:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:51:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:52:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:52:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:52:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:52:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:52:04 [WARNING ] (13, 1, 'bot')
2018-11-26 15:52:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:52:04 [DEBUG   ] Finished updating board flags in 1.560 seconds
2018-11-26 15:52:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:52:29 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:52:44 [DEBUG   ] Got command RPT from MCS: ref #634500
2018-11-26 15:52:44 [DEBUG   ] summary = ERROR
2018-11-26 15:52:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634500  13 58448 57164033 A  ERRORERROR'
2018-11-26 15:52:44 [DEBUG   ] Got command RPT from MCS: ref #634501
2018-11-26 15:52:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:52:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634501  77 58448 57164225 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:52:44 [DEBUG   ] Got command RPT from MCS: ref #634502
2018-11-26 15:52:44 [DEBUG   ] summary = ERROR
2018-11-26 15:52:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634502  13 58448 57164429 A  ERRORERROR'
2018-11-26 15:52:44 [DEBUG   ] Got command RPT from MCS: ref #634504
2018-11-26 15:52:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:52:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634504  77 58448 57164633 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:52:44 [DEBUG   ] Got command RPT from MCS: ref #634505
2018-11-26 15:52:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:52:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634505  12 58448 57164837 A  ERRORB—™š'
2018-11-26 15:52:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:53:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:53:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:53:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:53:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:53:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:53:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:53:05 [DEBUG   ] Finished updating board flags in 2.256 seconds
2018-11-26 15:53:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:53:29 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:53:49 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:54:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:54:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:54:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:54:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:54:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:54:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:54:05 [DEBUG   ] Finished updating board flags in 2.116 seconds
2018-11-26 15:54:09 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:54:29 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:54:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:55:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:55:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:55:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:55:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:55:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:55:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:55:05 [DEBUG   ] Finished updating board flags in 2.015 seconds
2018-11-26 15:55:09 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:55:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:55:48 [DEBUG   ] Got command RPT from MCS: ref #634514
2018-11-26 15:55:48 [DEBUG   ] summary = ERROR
2018-11-26 15:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634514  13 58448 57348737 A  ERRORERROR'
2018-11-26 15:55:48 [DEBUG   ] Got command RPT from MCS: ref #634515
2018-11-26 15:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634515  77 58448 57348738 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:55:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:56:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:56:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:56:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:56:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:56:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:56:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:56:05 [DEBUG   ] Finished updating board flags in 1.971 seconds
2018-11-26 15:56:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:56:29 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:56:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:57:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:57:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:57:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:57:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:57:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:57:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:57:05 [DEBUG   ] Finished updating board flags in 1.823 seconds
2018-11-26 15:57:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:57:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:57:44 [DEBUG   ] Got command RPT from MCS: ref #634531
2018-11-26 15:57:44 [DEBUG   ] summary = ERROR
2018-11-26 15:57:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634531  13 58448 57464105 A  ERRORERROR'
2018-11-26 15:57:44 [DEBUG   ] Got command RPT from MCS: ref #634532
2018-11-26 15:57:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:57:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634532  77 58448 57464309 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:57:44 [DEBUG   ] Got command RPT from MCS: ref #634533
2018-11-26 15:57:44 [DEBUG   ] summary = ERROR
2018-11-26 15:57:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634533  13 58448 57464513 A  ERRORERROR'
2018-11-26 15:57:44 [DEBUG   ] Got command RPT from MCS: ref #634534
2018-11-26 15:57:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 15:57:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634534  77 58448 57464717 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 15:57:44 [DEBUG   ] Got command RPT from MCS: ref #634535
2018-11-26 15:57:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 15:57:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634535  12 58448 57464909 A  ERRORB•™š'
2018-11-26 15:57:49 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:58:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:58:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:58:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:58:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:58:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:58:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:58:05 [DEBUG   ] Finished updating board flags in 1.880 seconds
2018-11-26 15:58:09 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:58:29 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:58:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 15:59:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 15:59:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:59:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:59:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:59:05 [WARNING ] (13, 1, 'bot')
2018-11-26 15:59:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 15:59:05 [DEBUG   ] Finished updating board flags in 1.717 seconds
2018-11-26 15:59:09 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 15:59:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 15:59:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:00:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:00:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:00:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:00:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:00:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:00:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:00:05 [DEBUG   ] Finished updating board flags in 1.633 seconds
2018-11-26 16:00:09 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 16:00:29 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 16:00:48 [DEBUG   ] Got command RPT from MCS: ref #634540
2018-11-26 16:00:48 [DEBUG   ] summary = ERROR
2018-11-26 16:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634540  13 58448 57648734 A  ERRORERROR'
2018-11-26 16:00:48 [DEBUG   ] Got command RPT from MCS: ref #634541
2018-11-26 16:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634541  77 58448 57648734 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:00:49 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:01:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 16:01:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:01:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:01:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:01:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:01:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:01:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:01:05 [DEBUG   ] Finished updating board flags in 1.570 seconds
2018-11-26 16:01:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:01:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:01:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:02:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:02:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:02:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:02:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:02:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:02:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:02:05 [DEBUG   ] Finished updating board flags in 1.412 seconds
2018-11-26 16:02:09 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:02:29 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:02:44 [DEBUG   ] Got command RPT from MCS: ref #634563
2018-11-26 16:02:44 [DEBUG   ] summary = ERROR
2018-11-26 16:02:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634563  13 58448 57764193 A  ERRORERROR'
2018-11-26 16:02:44 [DEBUG   ] Got command RPT from MCS: ref #634564
2018-11-26 16:02:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:02:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634564  77 58448 57764385 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:02:44 [DEBUG   ] Got command RPT from MCS: ref #634565
2018-11-26 16:02:44 [DEBUG   ] summary = ERROR
2018-11-26 16:02:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634565  13 58448 57764589 A  ERRORERROR'
2018-11-26 16:02:44 [DEBUG   ] Got command RPT from MCS: ref #634567
2018-11-26 16:02:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:02:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634567  77 58448 57764793 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:02:44 [DEBUG   ] Got command RPT from MCS: ref #634569
2018-11-26 16:02:44 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:02:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634569  12 58448 57764997 A  ERRORB—™š'
2018-11-26 16:02:49 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:03:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:03:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:03:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:03:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:03:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:03:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:03:05 [DEBUG   ] Finished updating board flags in 1.441 seconds
2018-11-26 16:03:09 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:03:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:03:49 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 16:04:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:04:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:04:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:04:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:04:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:04:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:04:05 [DEBUG   ] Finished updating board flags in 1.443 seconds
2018-11-26 16:04:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:04:29 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 16:04:49 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:05:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:05:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:05:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:05:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:05:05 [WARNING ] (13, 1, 'bot')
2018-11-26 16:05:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:05:05 [DEBUG   ] Finished updating board flags in 1.563 seconds
2018-11-26 16:05:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:05:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:05:48 [DEBUG   ] Got command RPT from MCS: ref #634577
2018-11-26 16:05:48 [DEBUG   ] summary = ERROR
2018-11-26 16:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634577  13 58448 57948737 A  ERRORERROR'
2018-11-26 16:05:48 [DEBUG   ] Got command RPT from MCS: ref #634578
2018-11-26 16:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634578  77 58448 57948738 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:05:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:06:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:06:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:06:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:06:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:06:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:06:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:06:06 [DEBUG   ] Finished updating board flags in 2.275 seconds
2018-11-26 16:06:09 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:06:29 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:06:49 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:07:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:07:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:07:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:07:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:07:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:07:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:07:06 [DEBUG   ] Finished updating board flags in 2.080 seconds
2018-11-26 16:07:09 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:07:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:07:44 [DEBUG   ] Got command RPT from MCS: ref #634598
2018-11-26 16:07:44 [DEBUG   ] summary = ERROR
2018-11-26 16:07:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634598  13 58448 58064265 A  ERRORERROR'
2018-11-26 16:07:44 [DEBUG   ] Got command RPT from MCS: ref #634599
2018-11-26 16:07:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:07:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634599  77 58448 58064469 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:07:44 [DEBUG   ] Got command RPT from MCS: ref #634600
2018-11-26 16:07:44 [DEBUG   ] summary = ERROR
2018-11-26 16:07:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634600  13 58448 58064673 A  ERRORERROR'
2018-11-26 16:07:44 [DEBUG   ] Got command RPT from MCS: ref #634601
2018-11-26 16:07:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:07:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634601  77 58448 58064877 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:07:45 [DEBUG   ] Got command RPT from MCS: ref #634602
2018-11-26 16:07:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:07:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634602  12 58448 58065069 A  ERRORB–™š'
2018-11-26 16:07:49 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:08:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:08:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:08:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:08:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:08:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:08:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:08:06 [DEBUG   ] Finished updating board flags in 2.002 seconds
2018-11-26 16:08:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:08:29 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:08:49 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:09:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:09:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:09:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:09:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:09:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:09:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:09:06 [DEBUG   ] Finished updating board flags in 2.008 seconds
2018-11-26 16:09:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:09:29 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:09:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:10:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:10:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:10:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:10:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:10:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:10:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:10:06 [DEBUG   ] Finished updating board flags in 2.008 seconds
2018-11-26 16:10:09 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 16:10:29 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:10:48 [DEBUG   ] Got command RPT from MCS: ref #634607
2018-11-26 16:10:48 [DEBUG   ] summary = ERROR
2018-11-26 16:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634607  13 58448 58248741 A  ERRORERROR'
2018-11-26 16:10:48 [DEBUG   ] Got command RPT from MCS: ref #634608
2018-11-26 16:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634608  77 58448 58248742 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:10:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:11:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:11:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:11:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:11:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:11:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:11:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:11:06 [DEBUG   ] Finished updating board flags in 1.976 seconds
2018-11-26 16:11:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:11:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:11:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:12:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:12:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:12:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:12:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:12:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:12:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:12:06 [DEBUG   ] Finished updating board flags in 1.843 seconds
2018-11-26 16:12:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:12:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:12:44 [DEBUG   ] Got command RPT from MCS: ref #634626
2018-11-26 16:12:44 [DEBUG   ] summary = ERROR
2018-11-26 16:12:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634626  13 58448 58364345 A  ERRORERROR'
2018-11-26 16:12:44 [DEBUG   ] Got command RPT from MCS: ref #634627
2018-11-26 16:12:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:12:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634627  77 58448 58364549 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:12:44 [DEBUG   ] Got command RPT from MCS: ref #634628
2018-11-26 16:12:44 [DEBUG   ] summary = ERROR
2018-11-26 16:12:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634628  13 58448 58364753 A  ERRORERROR'
2018-11-26 16:12:44 [DEBUG   ] Got command RPT from MCS: ref #634630
2018-11-26 16:12:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:12:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634630  77 58448 58364945 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:12:45 [DEBUG   ] Got command RPT from MCS: ref #634631
2018-11-26 16:12:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:12:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634631  12 58448 58365149 A  ERRORB•™š'
2018-11-26 16:12:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 16:13:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:13:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:13:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:13:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:13:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:13:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:13:06 [DEBUG   ] Finished updating board flags in 1.888 seconds
2018-11-26 16:13:09 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:13:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:13:49 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:14:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:14:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:14:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:14:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:14:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:14:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:14:06 [DEBUG   ] Finished updating board flags in 1.672 seconds
2018-11-26 16:14:09 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 16:14:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 16:14:49 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 16:15:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:15:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:15:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:15:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:15:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:15:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:15:06 [DEBUG   ] Finished updating board flags in 1.624 seconds
2018-11-26 16:15:09 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 16:15:29 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 16:15:48 [DEBUG   ] Got command RPT from MCS: ref #634640
2018-11-26 16:15:48 [DEBUG   ] summary = ERROR
2018-11-26 16:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634640  13 58448 58548738 A  ERRORERROR'
2018-11-26 16:15:48 [DEBUG   ] Got command RPT from MCS: ref #634641
2018-11-26 16:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634641  77 58448 58548739 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:15:49 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 16:16:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-26 16:16:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:16:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:16:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:16:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:16:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:16:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:16:06 [DEBUG   ] Finished updating board flags in 1.586 seconds
2018-11-26 16:16:09 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2018-11-26 16:16:29 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 16:16:50 [DEBUG   ] Finished updating ant stats in 1.393 seconds
2018-11-26 16:17:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:17:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:17:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:17:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:17:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:17:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:17:06 [DEBUG   ] Finished updating board flags in 1.411 seconds
2018-11-26 16:17:10 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 16:17:30 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 16:17:44 [DEBUG   ] Got command RPT from MCS: ref #634661
2018-11-26 16:17:44 [DEBUG   ] summary = ERROR
2018-11-26 16:17:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634661  13 58448 58664425 A  ERRORERROR'
2018-11-26 16:17:44 [DEBUG   ] Got command RPT from MCS: ref #634662
2018-11-26 16:17:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:17:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634662  77 58448 58664629 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:17:44 [DEBUG   ] Got command RPT from MCS: ref #634663
2018-11-26 16:17:44 [DEBUG   ] summary = ERROR
2018-11-26 16:17:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634663  13 58448 58664833 A  ERRORERROR'
2018-11-26 16:17:45 [DEBUG   ] Got command RPT from MCS: ref #634664
2018-11-26 16:17:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:17:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634664  77 58448 58665037 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:17:45 [DEBUG   ] Got command RPT from MCS: ref #634665
2018-11-26 16:17:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:17:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634665  12 58448 58665229 A  ERRORB–™š'
2018-11-26 16:17:50 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:18:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:18:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 16:18:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:18:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:18:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:18:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:18:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:18:06 [DEBUG   ] Finished updating board flags in 1.389 seconds
2018-11-26 16:18:10 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 16:18:30 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:18:50 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:19:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:19:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:19:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:19:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:19:06 [WARNING ] (13, 1, 'bot')
2018-11-26 16:19:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:19:06 [DEBUG   ] Finished updating board flags in 1.368 seconds
2018-11-26 16:19:10 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:19:30 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:19:50 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:20:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:20:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 16:20:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:20:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:20:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:20:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:20:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:20:07 [DEBUG   ] Finished updating board flags in 2.053 seconds
2018-11-26 16:20:10 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 16:20:30 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2018-11-26 16:20:48 [DEBUG   ] Got command RPT from MCS: ref #634670
2018-11-26 16:20:48 [DEBUG   ] summary = ERROR
2018-11-26 16:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634670  13 58448 58848741 A  ERRORERROR'
2018-11-26 16:20:48 [DEBUG   ] Got command RPT from MCS: ref #634671
2018-11-26 16:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634671  77 58448 58848742 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:20:50 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 16:21:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:21:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:21:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:21:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:21:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:21:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:21:07 [DEBUG   ] Finished updating board flags in 2.000 seconds
2018-11-26 16:21:10 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 16:21:30 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 16:21:50 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 16:22:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:22:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:22:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:22:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:22:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:22:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:22:07 [DEBUG   ] Finished updating board flags in 2.014 seconds
2018-11-26 16:22:10 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 16:22:30 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 16:22:44 [DEBUG   ] Got command RPT from MCS: ref #634693
2018-11-26 16:22:44 [DEBUG   ] summary = ERROR
2018-11-26 16:22:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634693  13 58448 58964509 A  ERRORERROR'
2018-11-26 16:22:44 [DEBUG   ] Got command RPT from MCS: ref #634694
2018-11-26 16:22:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:22:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634694  77 58448 58964713 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:22:44 [DEBUG   ] Got command RPT from MCS: ref #634695
2018-11-26 16:22:44 [DEBUG   ] summary = ERROR
2018-11-26 16:22:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634695  13 58448 58964905 A  ERRORERROR'
2018-11-26 16:22:45 [DEBUG   ] Got command RPT from MCS: ref #634697
2018-11-26 16:22:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:22:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634697  77 58448 58965109 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:22:45 [DEBUG   ] Got command RPT from MCS: ref #634699
2018-11-26 16:22:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:22:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634699  12 58448 58965313 A  ERRORB–™š'
2018-11-26 16:22:50 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 16:23:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:23:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:23:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:23:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:23:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:23:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:23:07 [DEBUG   ] Finished updating board flags in 1.981 seconds
2018-11-26 16:23:10 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 16:23:30 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 16:23:50 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 16:24:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:24:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 16:24:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:24:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:24:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:24:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:24:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:24:07 [DEBUG   ] Finished updating board flags in 1.809 seconds
2018-11-26 16:24:10 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 16:24:30 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 16:24:50 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 16:25:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:25:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 16:25:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:25:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:25:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:25:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:25:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:25:07 [DEBUG   ] Finished updating board flags in 1.859 seconds
2018-11-26 16:25:10 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 16:25:30 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 16:25:48 [DEBUG   ] Got command RPT from MCS: ref #634707
2018-11-26 16:25:48 [DEBUG   ] summary = ERROR
2018-11-26 16:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634707  13 58448 59148741 A  ERRORERROR'
2018-11-26 16:25:48 [DEBUG   ] Got command RPT from MCS: ref #634708
2018-11-26 16:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634708  77 58448 59148742 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:25:51 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 16:26:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:26:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:26:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:26:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:26:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:26:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:26:07 [DEBUG   ] Finished updating board flags in 1.810 seconds
2018-11-26 16:26:11 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 16:26:31 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:26:51 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 16:27:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:27:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:27:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:27:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:27:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:27:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:27:07 [DEBUG   ] Finished updating board flags in 1.725 seconds
2018-11-26 16:27:11 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 16:27:31 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 16:27:44 [DEBUG   ] Got command RPT from MCS: ref #634724
2018-11-26 16:27:44 [DEBUG   ] summary = ERROR
2018-11-26 16:27:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634724  13 58448 59264585 A  ERRORERROR'
2018-11-26 16:27:44 [DEBUG   ] Got command RPT from MCS: ref #634725
2018-11-26 16:27:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:27:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634725  77 58448 59264789 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:27:44 [DEBUG   ] Got command RPT from MCS: ref #634726
2018-11-26 16:27:44 [DEBUG   ] summary = ERROR
2018-11-26 16:27:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634726  13 58448 59264993 A  ERRORERROR'
2018-11-26 16:27:45 [DEBUG   ] Got command RPT from MCS: ref #634727
2018-11-26 16:27:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:27:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634727  77 58448 59265197 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:27:45 [DEBUG   ] Got command RPT from MCS: ref #634728
2018-11-26 16:27:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:27:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634728  12 58448 59265389 A  ERRORB–™š'
2018-11-26 16:27:51 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 16:28:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-26 16:28:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:28:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-26 16:28:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-26 16:28:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:28:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:28:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:28:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:28:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:28:07 [DEBUG   ] Finished updating board flags in 1.645 seconds
2018-11-26 16:28:11 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 16:28:31 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:28:51 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 16:29:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:29:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:29:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:29:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:29:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:29:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:29:07 [DEBUG   ] Finished updating board flags in 1.640 seconds
2018-11-26 16:29:11 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 16:29:31 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:29:51 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:30:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 16:30:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:30:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:30:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:30:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:30:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:30:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:30:07 [DEBUG   ] Finished updating board flags in 1.530 seconds
2018-11-26 16:30:11 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 16:30:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:30:48 [DEBUG   ] Got command RPT from MCS: ref #634733
2018-11-26 16:30:48 [DEBUG   ] summary = ERROR
2018-11-26 16:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634733  13 58448 59448745 A  ERRORERROR'
2018-11-26 16:30:48 [DEBUG   ] Got command RPT from MCS: ref #634734
2018-11-26 16:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634734  77 58448 59448746 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:30:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:31:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:31:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:31:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:31:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:31:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:31:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:31:07 [DEBUG   ] Finished updating board flags in 1.420 seconds
2018-11-26 16:31:11 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:31:31 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:31:51 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:32:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:32:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 16:32:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:32:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:32:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:32:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:32:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:32:07 [DEBUG   ] Finished updating board flags in 1.540 seconds
2018-11-26 16:32:11 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:32:31 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:32:44 [DEBUG   ] Got command RPT from MCS: ref #634756
2018-11-26 16:32:44 [DEBUG   ] summary = ERROR
2018-11-26 16:32:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634756  13 58448 59564673 A  ERRORERROR'
2018-11-26 16:32:44 [DEBUG   ] Got command RPT from MCS: ref #634757
2018-11-26 16:32:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:32:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634757  77 58448 59564877 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:32:45 [DEBUG   ] Got command RPT from MCS: ref #634758
2018-11-26 16:32:45 [DEBUG   ] summary = ERROR
2018-11-26 16:32:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634758  13 58448 59565069 A  ERRORERROR'
2018-11-26 16:32:45 [DEBUG   ] Got command RPT from MCS: ref #634760
2018-11-26 16:32:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:32:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634760  77 58448 59565273 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:32:45 [DEBUG   ] Got command RPT from MCS: ref #634762
2018-11-26 16:32:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:32:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634762  12 58448 59565477 A  ERRORB—™š'
2018-11-26 16:32:51 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:33:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:33:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:33:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:33:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:33:07 [WARNING ] (13, 1, 'bot')
2018-11-26 16:33:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:33:07 [DEBUG   ] Finished updating board flags in 1.561 seconds
2018-11-26 16:33:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:33:31 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:33:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:34:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:34:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:34:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:34:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:34:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:34:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:34:08 [DEBUG   ] Finished updating board flags in 2.168 seconds
2018-11-26 16:34:11 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:34:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:34:51 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 16:35:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:35:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:35:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:35:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:35:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:35:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:35:08 [DEBUG   ] Finished updating board flags in 1.958 seconds
2018-11-26 16:35:11 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:35:31 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 16:35:48 [DEBUG   ] Got command RPT from MCS: ref #634770
2018-11-26 16:35:48 [DEBUG   ] summary = ERROR
2018-11-26 16:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634770  13 58448 59748749 A  ERRORERROR'
2018-11-26 16:35:48 [DEBUG   ] Got command RPT from MCS: ref #634771
2018-11-26 16:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634771  77 58448 59748750 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:35:51 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:36:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:36:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:36:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:36:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:36:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:36:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:36:08 [DEBUG   ] Finished updating board flags in 1.774 seconds
2018-11-26 16:36:11 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:36:31 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:36:51 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:37:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:37:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:37:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:37:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:37:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:37:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:37:08 [DEBUG   ] Finished updating board flags in 1.611 seconds
2018-11-26 16:37:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:37:31 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:37:44 [DEBUG   ] Got command RPT from MCS: ref #634791
2018-11-26 16:37:44 [DEBUG   ] summary = ERROR
2018-11-26 16:37:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634791  13 58448 59864749 A  ERRORERROR'
2018-11-26 16:37:44 [DEBUG   ] Got command RPT from MCS: ref #634792
2018-11-26 16:37:44 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:37:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634792  77 58448 59864953 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:37:45 [DEBUG   ] Got command RPT from MCS: ref #634793
2018-11-26 16:37:45 [DEBUG   ] summary = ERROR
2018-11-26 16:37:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634793  13 58448 59865157 A  ERRORERROR'
2018-11-26 16:37:45 [DEBUG   ] Got command RPT from MCS: ref #634794
2018-11-26 16:37:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:37:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634794  77 58448 59865361 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:37:45 [DEBUG   ] Got command RPT from MCS: ref #634795
2018-11-26 16:37:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:37:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634795  12 58448 59865553 A  ERRORB–™š'
2018-11-26 16:37:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:38:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:38:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:38:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:38:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:38:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:38:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:38:08 [DEBUG   ] Finished updating board flags in 1.575 seconds
2018-11-26 16:38:11 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 16:38:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:38:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:39:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:39:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 16:39:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:39:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:39:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:39:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:39:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:39:08 [DEBUG   ] Finished updating board flags in 1.414 seconds
2018-11-26 16:39:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:39:31 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:39:51 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 16:40:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:40:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:40:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:40:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:40:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:40:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:40:08 [DEBUG   ] Finished updating board flags in 1.546 seconds
2018-11-26 16:40:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:40:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:40:48 [DEBUG   ] Got command RPT from MCS: ref #634800
2018-11-26 16:40:48 [DEBUG   ] summary = ERROR
2018-11-26 16:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634800  13 58448 60048741 A  ERRORERROR'
2018-11-26 16:40:48 [DEBUG   ] Got command RPT from MCS: ref #634801
2018-11-26 16:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634801  77 58448 60048742 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:40:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:41:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:41:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:41:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:41:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:41:08 [WARNING ] (13, 1, 'bot')
2018-11-26 16:41:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:41:08 [DEBUG   ] Finished updating board flags in 1.586 seconds
2018-11-26 16:41:11 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 16:41:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:41:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:42:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:42:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:42:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:42:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:42:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:42:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:42:09 [DEBUG   ] Finished updating board flags in 2.189 seconds
2018-11-26 16:42:11 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:42:31 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 16:42:44 [DEBUG   ] Got command RPT from MCS: ref #634819
2018-11-26 16:42:44 [DEBUG   ] summary = ERROR
2018-11-26 16:42:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634819  13 58448 60164829 A  ERRORERROR'
2018-11-26 16:42:45 [DEBUG   ] Got command RPT from MCS: ref #634820
2018-11-26 16:42:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:42:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634820  77 58448 60165033 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:42:45 [DEBUG   ] Got command RPT from MCS: ref #634821
2018-11-26 16:42:45 [DEBUG   ] summary = ERROR
2018-11-26 16:42:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634821  13 58448 60165237 A  ERRORERROR'
2018-11-26 16:42:45 [DEBUG   ] Got command RPT from MCS: ref #634823
2018-11-26 16:42:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:42:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634823  77 58448 60165441 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:42:45 [DEBUG   ] Got command RPT from MCS: ref #634824
2018-11-26 16:42:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:42:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634824  12 58448 60165633 A  ERRORB•™š'
2018-11-26 16:42:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:43:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 16:43:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:43:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:43:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:43:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:43:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:43:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:43:09 [DEBUG   ] Finished updating board flags in 1.869 seconds
2018-11-26 16:43:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:43:31 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:43:51 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 16:44:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:44:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:44:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:44:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:44:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:44:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:44:09 [DEBUG   ] Finished updating board flags in 1.828 seconds
2018-11-26 16:44:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:44:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:44:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:45:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:45:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:45:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:45:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:45:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:45:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:45:09 [DEBUG   ] Finished updating board flags in 1.781 seconds
2018-11-26 16:45:11 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 16:45:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:45:48 [DEBUG   ] Got command RPT from MCS: ref #634833
2018-11-26 16:45:48 [DEBUG   ] summary = ERROR
2018-11-26 16:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634833  13 58448 60348749 A  ERRORERROR'
2018-11-26 16:45:48 [DEBUG   ] Got command RPT from MCS: ref #634834
2018-11-26 16:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634834  77 58448 60348750 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:45:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:46:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:46:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 16:46:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:46:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:46:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:46:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:46:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:46:09 [DEBUG   ] Finished updating board flags in 1.621 seconds
2018-11-26 16:46:11 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 16:46:31 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 16:46:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:47:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa16, chip 1
2018-11-26 16:47:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 16:47:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:47:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:47:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:47:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:47:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:47:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:47:09 [DEBUG   ] Finished updating board flags in 1.692 seconds
2018-11-26 16:47:11 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 16:47:31 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:47:44 [DEBUG   ] Got command RPT from MCS: ref #634854
2018-11-26 16:47:44 [DEBUG   ] summary = ERROR
2018-11-26 16:47:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634854  13 58448 60464909 A  ERRORERROR'
2018-11-26 16:47:45 [DEBUG   ] Got command RPT from MCS: ref #634855
2018-11-26 16:47:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:47:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634855  77 58448 60465113 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:47:45 [DEBUG   ] Got command RPT from MCS: ref #634856
2018-11-26 16:47:45 [DEBUG   ] summary = ERROR
2018-11-26 16:47:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634856  13 58448 60465317 A  ERRORERROR'
2018-11-26 16:47:45 [DEBUG   ] Got command RPT from MCS: ref #634857
2018-11-26 16:47:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:47:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634857  77 58448 60465521 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:47:45 [DEBUG   ] Got command RPT from MCS: ref #634858
2018-11-26 16:47:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:47:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634858  12 58448 60465713 A  ERRORB—™š'
2018-11-26 16:47:51 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 16:48:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:48:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:48:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:48:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:48:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:48:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:48:09 [DEBUG   ] Finished updating board flags in 1.616 seconds
2018-11-26 16:48:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:48:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:48:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 16:49:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:49:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:49:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:49:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:49:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:49:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:49:09 [DEBUG   ] Finished updating board flags in 1.481 seconds
2018-11-26 16:49:11 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 16:49:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:49:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 16:50:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:50:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:50:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:50:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:50:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:50:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:50:09 [DEBUG   ] Finished updating board flags in 1.492 seconds
2018-11-26 16:50:11 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2018-11-26 16:50:31 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 16:50:48 [DEBUG   ] Got command RPT from MCS: ref #634863
2018-11-26 16:50:48 [DEBUG   ] summary = ERROR
2018-11-26 16:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634863  13 58448 60648753 A  ERRORERROR'
2018-11-26 16:50:48 [DEBUG   ] Got command RPT from MCS: ref #634864
2018-11-26 16:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634864  77 58448 60648754 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:50:52 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 16:51:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:51:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:51:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:51:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:51:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:51:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:51:09 [DEBUG   ] Finished updating board flags in 1.492 seconds
2018-11-26 16:51:12 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 16:51:32 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 16:51:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:52:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:52:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:52:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:52:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:52:09 [WARNING ] (13, 1, 'bot')
2018-11-26 16:52:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:52:09 [DEBUG   ] Finished updating board flags in 1.487 seconds
2018-11-26 16:52:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:52:32 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 16:52:44 [DEBUG   ] Got command RPT from MCS: ref #634886
2018-11-26 16:52:44 [DEBUG   ] summary = ERROR
2018-11-26 16:52:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634886  13 58448 60764997 A  ERRORERROR'
2018-11-26 16:52:45 [DEBUG   ] Got command RPT from MCS: ref #634887
2018-11-26 16:52:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:52:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634887  77 58448 60765189 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:52:45 [DEBUG   ] Got command RPT from MCS: ref #634888
2018-11-26 16:52:45 [DEBUG   ] summary = ERROR
2018-11-26 16:52:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634888  13 58448 60765393 A  ERRORERROR'
2018-11-26 16:52:45 [DEBUG   ] Got command RPT from MCS: ref #634890
2018-11-26 16:52:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:52:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634890  77 58448 60765597 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:52:45 [DEBUG   ] Got command RPT from MCS: ref #634892
2018-11-26 16:52:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:52:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634892  12 58448 60765801 A  ERRORB•™š'
2018-11-26 16:52:52 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 16:53:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:53:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:53:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:53:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:53:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:53:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:53:10 [DEBUG   ] Finished updating board flags in 2.157 seconds
2018-11-26 16:53:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 16:53:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:53:52 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 16:54:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:54:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:54:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:54:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:54:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:54:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:54:10 [DEBUG   ] Finished updating board flags in 2.015 seconds
2018-11-26 16:54:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 16:54:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:54:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:55:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:55:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:55:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:55:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:55:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:55:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:55:10 [DEBUG   ] Finished updating board flags in 2.035 seconds
2018-11-26 16:55:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 16:55:32 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 16:55:48 [DEBUG   ] Got command RPT from MCS: ref #634900
2018-11-26 16:55:48 [DEBUG   ] summary = ERROR
2018-11-26 16:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634900  13 58448 60948753 A  ERRORERROR'
2018-11-26 16:55:48 [DEBUG   ] Got command RPT from MCS: ref #634901
2018-11-26 16:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634901  77 58448 60948754 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:55:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:56:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:56:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:56:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:56:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:56:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:56:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:56:10 [DEBUG   ] Finished updating board flags in 2.056 seconds
2018-11-26 16:56:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 16:56:32 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 16:56:52 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 16:57:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:57:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:57:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:57:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:57:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:57:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:57:10 [DEBUG   ] Finished updating board flags in 1.879 seconds
2018-11-26 16:57:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:57:32 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:57:45 [DEBUG   ] Got command RPT from MCS: ref #634917
2018-11-26 16:57:45 [DEBUG   ] summary = ERROR
2018-11-26 16:57:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634917  13 58448 61065073 A  ERRORERROR'
2018-11-26 16:57:45 [DEBUG   ] Got command RPT from MCS: ref #634918
2018-11-26 16:57:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:57:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634918  77 58448 61065277 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:57:45 [DEBUG   ] Got command RPT from MCS: ref #634919
2018-11-26 16:57:45 [DEBUG   ] summary = ERROR
2018-11-26 16:57:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634919  13 58448 61065481 A  ERRORERROR'
2018-11-26 16:57:45 [DEBUG   ] Got command RPT from MCS: ref #634920
2018-11-26 16:57:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 16:57:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634920  77 58448 61065673 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 16:57:45 [DEBUG   ] Got command RPT from MCS: ref #634921
2018-11-26 16:57:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 16:57:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634921  12 58448 61065877 A  ERRORB—™š'
2018-11-26 16:57:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:58:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:58:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-26 16:58:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:58:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:58:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:58:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:58:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:58:10 [DEBUG   ] Finished updating board flags in 1.820 seconds
2018-11-26 16:58:12 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:58:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 16:58:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 16:59:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 16:59:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:59:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:59:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:59:10 [WARNING ] (13, 1, 'bot')
2018-11-26 16:59:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 16:59:10 [DEBUG   ] Finished updating board flags in 1.794 seconds
2018-11-26 16:59:12 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 16:59:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 16:59:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:00:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:00:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:00:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:00:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:00:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:00:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:00:10 [DEBUG   ] Finished updating board flags in 1.622 seconds
2018-11-26 17:00:12 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:00:32 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:00:48 [DEBUG   ] Got command RPT from MCS: ref #634926
2018-11-26 17:00:48 [DEBUG   ] summary = ERROR
2018-11-26 17:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634926  13 58448 61248753 A  ERRORERROR'
2018-11-26 17:00:48 [DEBUG   ] Got command RPT from MCS: ref #634927
2018-11-26 17:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634927  77 58448 61248754 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:00:52 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:01:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:01:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:01:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:01:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:01:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:01:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:01:10 [DEBUG   ] Finished updating board flags in 1.660 seconds
2018-11-26 17:01:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:01:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:01:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:02:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:02:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 17:02:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 17:02:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:02:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:02:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:02:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:02:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:02:10 [DEBUG   ] Finished updating board flags in 1.539 seconds
2018-11-26 17:02:12 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:02:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:02:45 [DEBUG   ] Got command RPT from MCS: ref #634949
2018-11-26 17:02:45 [DEBUG   ] summary = ERROR
2018-11-26 17:02:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634949  13 58448 61365153 A  ERRORERROR'
2018-11-26 17:02:45 [DEBUG   ] Got command RPT from MCS: ref #634950
2018-11-26 17:02:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:02:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634950  77 58448 61365357 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:02:45 [DEBUG   ] Got command RPT from MCS: ref #634951
2018-11-26 17:02:45 [DEBUG   ] summary = ERROR
2018-11-26 17:02:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634951  13 58448 61365561 A  ERRORERROR'
2018-11-26 17:02:45 [DEBUG   ] Got command RPT from MCS: ref #634953
2018-11-26 17:02:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:02:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634953  77 58448 61365753 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:02:45 [DEBUG   ] Got command RPT from MCS: ref #634955
2018-11-26 17:02:45 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:02:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634955  12 58448 61365957 A  ERRORB—™š'
2018-11-26 17:02:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:03:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:03:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:03:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:03:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:03:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:03:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:03:10 [DEBUG   ] Finished updating board flags in 1.518 seconds
2018-11-26 17:03:12 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:03:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:03:52 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:04:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:04:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 17:04:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:04:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:04:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:04:10 [WARNING ] (13, 1, 'bot')
2018-11-26 17:04:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:04:10 [DEBUG   ] Finished updating board flags in 1.532 seconds
2018-11-26 17:04:12 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:04:32 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:04:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:05:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:05:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:05:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:05:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:05:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:05:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:05:11 [DEBUG   ] Finished updating board flags in 2.217 seconds
2018-11-26 17:05:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:05:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:05:48 [DEBUG   ] Got command RPT from MCS: ref #634963
2018-11-26 17:05:48 [DEBUG   ] summary = ERROR
2018-11-26 17:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634963  13 58448 61548753 A  ERRORERROR'
2018-11-26 17:05:48 [DEBUG   ] Got command RPT from MCS: ref #634964
2018-11-26 17:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634964  77 58448 61548754 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:05:52 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 17:06:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:06:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:06:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:06:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:06:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:06:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:06:11 [DEBUG   ] Finished updating board flags in 2.258 seconds
2018-11-26 17:06:12 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:06:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:06:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:07:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:07:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:07:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:07:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:07:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:07:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:07:11 [DEBUG   ] Finished updating board flags in 2.132 seconds
2018-11-26 17:07:12 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 17:07:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:07:45 [DEBUG   ] Got command RPT from MCS: ref #634984
2018-11-26 17:07:45 [DEBUG   ] summary = ERROR
2018-11-26 17:07:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634984  13 58448 61665233 A  ERRORERROR'
2018-11-26 17:07:45 [DEBUG   ] Got command RPT from MCS: ref #634985
2018-11-26 17:07:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:07:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634985  77 58448 61665437 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:07:45 [DEBUG   ] Got command RPT from MCS: ref #634986
2018-11-26 17:07:45 [DEBUG   ] summary = ERROR
2018-11-26 17:07:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634986  13 58448 61665641 A  ERRORERROR'
2018-11-26 17:07:45 [DEBUG   ] Got command RPT from MCS: ref #634987
2018-11-26 17:07:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:07:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634987  77 58448 61665833 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:07:46 [DEBUG   ] Got command RPT from MCS: ref #634988
2018-11-26 17:07:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:07:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634988  12 58448 61666037 A  ERRORB•™š'
2018-11-26 17:07:52 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:08:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:08:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:08:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:08:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:08:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:08:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:08:11 [DEBUG   ] Finished updating board flags in 1.998 seconds
2018-11-26 17:08:12 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:08:32 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:08:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:09:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:09:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:09:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:09:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:09:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:09:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:09:11 [DEBUG   ] Finished updating board flags in 1.776 seconds
2018-11-26 17:09:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:09:32 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:09:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:10:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:10:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:10:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:10:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:10:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:10:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:10:11 [DEBUG   ] Finished updating board flags in 1.617 seconds
2018-11-26 17:10:12 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:10:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:10:48 [DEBUG   ] Got command RPT from MCS: ref #634993
2018-11-26 17:10:48 [DEBUG   ] summary = ERROR
2018-11-26 17:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634993  13 58448 61848753 A  ERRORERROR'
2018-11-26 17:10:48 [DEBUG   ] Got command RPT from MCS: ref #634994
2018-11-26 17:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   634994  77 58448 61848754 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:10:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:11:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:11:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:11:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:11:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:11:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:11:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:11:11 [DEBUG   ] Finished updating board flags in 1.679 seconds
2018-11-26 17:11:12 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:11:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:11:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:12:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-26 17:12:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:12:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:12:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:12:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:12:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:12:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:12:11 [DEBUG   ] Finished updating board flags in 1.535 seconds
2018-11-26 17:12:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:12:32 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:12:45 [DEBUG   ] Got command RPT from MCS: ref #635012
2018-11-26 17:12:45 [DEBUG   ] summary = ERROR
2018-11-26 17:12:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635012  13 58448 61965313 A  ERRORERROR'
2018-11-26 17:12:45 [DEBUG   ] Got command RPT from MCS: ref #635013
2018-11-26 17:12:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:12:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635013  77 58448 61965517 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:12:45 [DEBUG   ] Got command RPT from MCS: ref #635014
2018-11-26 17:12:45 [DEBUG   ] summary = ERROR
2018-11-26 17:12:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635014  13 58448 61965713 A  ERRORERROR'
2018-11-26 17:12:45 [DEBUG   ] Got command RPT from MCS: ref #635016
2018-11-26 17:12:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:12:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635016  77 58448 61965917 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:12:46 [DEBUG   ] Got command RPT from MCS: ref #635017
2018-11-26 17:12:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:12:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635017  12 58448 61966121 A  ERRORB–™š'
2018-11-26 17:12:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:13:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 17:13:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:13:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:13:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:13:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:13:11 [WARNING ] (13, 1, 'bot')
2018-11-26 17:13:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:13:11 [DEBUG   ] Finished updating board flags in 1.572 seconds
2018-11-26 17:13:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:13:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:13:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:14:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:14:12 [DEBUG   ] Finished updating ant stats in 0.950 seconds
2018-11-26 17:14:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:14:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:14:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:14:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:14:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:14:12 [DEBUG   ] Finished updating board flags in 2.278 seconds
2018-11-26 17:14:32 [DEBUG   ] Finished updating ant stats in 0.757 seconds
2018-11-26 17:14:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 17:15:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:15:12 [DEBUG   ] Finished updating ant stats in 0.643 seconds
2018-11-26 17:15:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:15:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:15:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:15:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:15:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:15:12 [DEBUG   ] Finished updating board flags in 2.210 seconds
2018-11-26 17:15:32 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2018-11-26 17:15:48 [DEBUG   ] Got command RPT from MCS: ref #635026
2018-11-26 17:15:48 [DEBUG   ] summary = ERROR
2018-11-26 17:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635026  13 58448 62148757 A  ERRORERROR'
2018-11-26 17:15:48 [DEBUG   ] Got command RPT from MCS: ref #635027
2018-11-26 17:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635027  77 58448 62148758 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:15:53 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 17:16:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:16:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:16:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:16:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:16:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:16:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:16:12 [DEBUG   ] Finished updating board flags in 2.123 seconds
2018-11-26 17:16:13 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 17:16:33 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:16:53 [DEBUG   ] Finished updating ant stats in 1.185 seconds
2018-11-26 17:17:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:17:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 17:17:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:17:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:17:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:17:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:17:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:17:12 [DEBUG   ] Finished updating board flags in 2.039 seconds
2018-11-26 17:17:13 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:17:33 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:17:45 [DEBUG   ] Got command RPT from MCS: ref #635047
2018-11-26 17:17:45 [DEBUG   ] summary = ERROR
2018-11-26 17:17:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635047  13 58448 62265405 A  ERRORERROR'
2018-11-26 17:17:45 [DEBUG   ] Got command RPT from MCS: ref #635048
2018-11-26 17:17:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:17:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635048  77 58448 62265597 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:17:45 [DEBUG   ] Got command RPT from MCS: ref #635049
2018-11-26 17:17:45 [DEBUG   ] summary = ERROR
2018-11-26 17:17:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635049  13 58448 62265801 A  ERRORERROR'
2018-11-26 17:17:46 [DEBUG   ] Got command RPT from MCS: ref #635050
2018-11-26 17:17:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:17:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635050  77 58448 62266005 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:17:46 [DEBUG   ] Got command RPT from MCS: ref #635051
2018-11-26 17:17:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:17:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635051  12 58448 62266197 A  ERRORB—™š'
2018-11-26 17:17:53 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:18:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:18:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:18:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:18:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:18:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:18:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:18:12 [DEBUG   ] Finished updating board flags in 1.969 seconds
2018-11-26 17:18:13 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:18:33 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:18:53 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 17:19:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:19:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:19:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:19:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:19:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:19:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:19:12 [DEBUG   ] Finished updating board flags in 1.822 seconds
2018-11-26 17:19:13 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 17:19:33 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 17:19:53 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 17:20:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:20:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:20:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:20:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:20:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:20:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:20:12 [DEBUG   ] Finished updating board flags in 1.795 seconds
2018-11-26 17:20:13 [DEBUG   ] Finished updating ant stats in 0.807 seconds
2018-11-26 17:20:33 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 17:20:48 [DEBUG   ] Got command RPT from MCS: ref #635056
2018-11-26 17:20:48 [DEBUG   ] summary = ERROR
2018-11-26 17:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635056  13 58448 62448761 A  ERRORERROR'
2018-11-26 17:20:48 [DEBUG   ] Got command RPT from MCS: ref #635057
2018-11-26 17:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635057  77 58448 62448762 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:20:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 17:21:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:21:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:21:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:21:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:21:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:21:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:21:12 [DEBUG   ] Finished updating board flags in 1.585 seconds
2018-11-26 17:21:13 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 17:21:34 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 17:21:54 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 17:22:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:22:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:22:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:22:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:22:12 [WARNING ] (13, 1, 'bot')
2018-11-26 17:22:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:22:12 [DEBUG   ] Finished updating board flags in 1.377 seconds
2018-11-26 17:22:14 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:22:34 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 17:22:45 [DEBUG   ] Got command RPT from MCS: ref #635079
2018-11-26 17:22:45 [DEBUG   ] summary = ERROR
2018-11-26 17:22:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635079  13 58448 62565485 A  ERRORERROR'
2018-11-26 17:22:45 [DEBUG   ] Got command RPT from MCS: ref #635080
2018-11-26 17:22:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:22:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635080  77 58448 62565677 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:22:45 [DEBUG   ] Got command RPT from MCS: ref #635081
2018-11-26 17:22:45 [DEBUG   ] summary = ERROR
2018-11-26 17:22:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635081  13 58448 62565881 A  ERRORERROR'
2018-11-26 17:22:46 [DEBUG   ] Got command RPT from MCS: ref #635083
2018-11-26 17:22:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:22:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635083  77 58448 62566085 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:22:46 [DEBUG   ] Got command RPT from MCS: ref #635084
2018-11-26 17:22:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:22:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635084  12 58448 62566289 A  ERRORB•™š'
2018-11-26 17:22:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 17:23:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:23:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:23:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:23:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:23:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:23:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:23:13 [DEBUG   ] Finished updating board flags in 2.296 seconds
2018-11-26 17:23:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:23:34 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:23:54 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 17:24:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:24:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 17:24:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:24:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:24:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:24:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:24:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:24:13 [DEBUG   ] Finished updating board flags in 2.259 seconds
2018-11-26 17:24:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 17:24:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:24:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:25:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:25:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:25:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:25:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:25:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:25:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:25:13 [DEBUG   ] Finished updating board flags in 2.082 seconds
2018-11-26 17:25:14 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:25:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 17:25:48 [DEBUG   ] Got command RPT from MCS: ref #635093
2018-11-26 17:25:48 [DEBUG   ] summary = ERROR
2018-11-26 17:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635093  13 58448 62748753 A  ERRORERROR'
2018-11-26 17:25:48 [DEBUG   ] Got command RPT from MCS: ref #635094
2018-11-26 17:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635094  77 58448 62748754 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:25:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:26:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:26:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:26:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:26:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:26:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:26:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:26:13 [DEBUG   ] Finished updating board flags in 2.022 seconds
2018-11-26 17:26:14 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:26:34 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:26:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 17:27:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 17:27:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:27:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:27:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:27:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:27:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:27:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:27:13 [DEBUG   ] Finished updating board flags in 1.987 seconds
2018-11-26 17:27:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:27:34 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:27:45 [DEBUG   ] Got command RPT from MCS: ref #635110
2018-11-26 17:27:45 [DEBUG   ] summary = ERROR
2018-11-26 17:27:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635110  13 58448 62865557 A  ERRORERROR'
2018-11-26 17:27:45 [DEBUG   ] Got command RPT from MCS: ref #635111
2018-11-26 17:27:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:27:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635111  77 58448 62865761 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:27:45 [DEBUG   ] Got command RPT from MCS: ref #635112
2018-11-26 17:27:45 [DEBUG   ] summary = ERROR
2018-11-26 17:27:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635112  13 58448 62865965 A  ERRORERROR'
2018-11-26 17:27:46 [DEBUG   ] Got command RPT from MCS: ref #635113
2018-11-26 17:27:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:27:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635113  77 58448 62866157 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:27:46 [DEBUG   ] Got command RPT from MCS: ref #635114
2018-11-26 17:27:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:27:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635114  12 58448 62866361 A  ERRORB–™š'
2018-11-26 17:27:54 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:28:12 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 17:28:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:28:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:28:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:28:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:28:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:28:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:28:13 [DEBUG   ] Finished updating board flags in 1.819 seconds
2018-11-26 17:28:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:28:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:28:54 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:29:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:29:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:29:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:29:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:29:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:29:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:29:13 [DEBUG   ] Finished updating board flags in 1.838 seconds
2018-11-26 17:29:14 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 17:29:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:29:54 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:30:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:30:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:30:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:30:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:30:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:30:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:30:13 [DEBUG   ] Finished updating board flags in 1.745 seconds
2018-11-26 17:30:14 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:30:34 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:30:48 [DEBUG   ] Got command RPT from MCS: ref #635119
2018-11-26 17:30:48 [DEBUG   ] summary = ERROR
2018-11-26 17:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635119  13 58448 63048753 A  ERRORERROR'
2018-11-26 17:30:48 [DEBUG   ] Got command RPT from MCS: ref #635120
2018-11-26 17:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635120  77 58448 63048754 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:30:54 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:31:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:31:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:31:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:31:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:31:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:31:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:31:13 [DEBUG   ] Finished updating board flags in 1.671 seconds
2018-11-26 17:31:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:31:34 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:31:54 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 17:32:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:32:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:32:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:32:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:32:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:32:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:32:13 [DEBUG   ] Finished updating board flags in 1.527 seconds
2018-11-26 17:32:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:32:34 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:32:45 [DEBUG   ] Got command RPT from MCS: ref #635142
2018-11-26 17:32:45 [DEBUG   ] summary = ERROR
2018-11-26 17:32:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635142  13 58448 63165645 A  ERRORERROR'
2018-11-26 17:32:45 [DEBUG   ] Got command RPT from MCS: ref #635143
2018-11-26 17:32:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:32:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635143  77 58448 63165837 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:32:46 [DEBUG   ] Got command RPT from MCS: ref #635144
2018-11-26 17:32:46 [DEBUG   ] summary = ERROR
2018-11-26 17:32:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635144  13 58448 63166041 A  ERRORERROR'
2018-11-26 17:32:46 [DEBUG   ] Got command RPT from MCS: ref #635146
2018-11-26 17:32:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:32:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635146  77 58448 63166245 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:32:46 [DEBUG   ] Got command RPT from MCS: ref #635147
2018-11-26 17:32:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:32:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635147  12 58448 63166449 A  ERRORB–™š'
2018-11-26 17:32:54 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:33:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:33:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:33:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:33:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:33:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:33:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:33:13 [DEBUG   ] Finished updating board flags in 1.510 seconds
2018-11-26 17:33:14 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 17:33:34 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 17:33:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 17:34:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:34:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:34:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:34:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:34:13 [WARNING ] (13, 1, 'bot')
2018-11-26 17:34:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:34:13 [DEBUG   ] Finished updating board flags in 1.526 seconds
2018-11-26 17:34:14 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 17:34:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 17:34:54 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 17:35:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:35:14 [DEBUG   ] Finished updating ant stats in 0.854 seconds
2018-11-26 17:35:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:35:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:35:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:35:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:35:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:35:14 [DEBUG   ] Finished updating board flags in 2.228 seconds
2018-11-26 17:35:34 [DEBUG   ] Finished updating ant stats in 0.751 seconds
2018-11-26 17:35:48 [DEBUG   ] Got command RPT from MCS: ref #635156
2018-11-26 17:35:48 [DEBUG   ] summary = ERROR
2018-11-26 17:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635156  13 58448 63348761 A  ERRORERROR'
2018-11-26 17:35:48 [DEBUG   ] Got command RPT from MCS: ref #635157
2018-11-26 17:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635157  77 58448 63348762 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:35:54 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 17:36:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:36:14 [DEBUG   ] Finished updating ant stats in 0.645 seconds
2018-11-26 17:36:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:36:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:36:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:36:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:36:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:36:14 [DEBUG   ] Finished updating board flags in 2.157 seconds
2018-11-26 17:36:34 [DEBUG   ] Finished updating ant stats in 0.558 seconds
2018-11-26 17:36:55 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 17:37:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:37:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:37:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:37:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:37:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:37:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:37:14 [DEBUG   ] Finished updating board flags in 2.021 seconds
2018-11-26 17:37:15 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 17:37:35 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:37:45 [DEBUG   ] Got command RPT from MCS: ref #635177
2018-11-26 17:37:45 [DEBUG   ] summary = ERROR
2018-11-26 17:37:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635177  13 58448 63465717 A  ERRORERROR'
2018-11-26 17:37:45 [DEBUG   ] Got command RPT from MCS: ref #635178
2018-11-26 17:37:45 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:37:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635178  77 58448 63465921 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:37:46 [DEBUG   ] Got command RPT from MCS: ref #635179
2018-11-26 17:37:46 [DEBUG   ] summary = ERROR
2018-11-26 17:37:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635179  13 58448 63466125 A  ERRORERROR'
2018-11-26 17:37:46 [DEBUG   ] Got command RPT from MCS: ref #635180
2018-11-26 17:37:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:37:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635180  77 58448 63466329 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:37:46 [DEBUG   ] Got command RPT from MCS: ref #635181
2018-11-26 17:37:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:37:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635181  12 58448 63466521 A  ERRORB•™š'
2018-11-26 17:37:55 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:38:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:38:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 17:38:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:38:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:38:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:38:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:38:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:38:14 [DEBUG   ] Finished updating board flags in 2.006 seconds
2018-11-26 17:38:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:38:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:38:55 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 17:39:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:39:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:39:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:39:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:39:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:39:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:39:14 [DEBUG   ] Finished updating board flags in 2.043 seconds
2018-11-26 17:39:15 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 17:39:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:39:55 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 17:40:13 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 17:40:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:40:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:40:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:40:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:40:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:40:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:40:14 [DEBUG   ] Finished updating board flags in 1.970 seconds
2018-11-26 17:40:15 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 17:40:35 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 17:40:48 [DEBUG   ] Got command RPT from MCS: ref #635186
2018-11-26 17:40:48 [DEBUG   ] summary = ERROR
2018-11-26 17:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635186  13 58448 63648757 A  ERRORERROR'
2018-11-26 17:40:48 [DEBUG   ] Got command RPT from MCS: ref #635187
2018-11-26 17:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635187  77 58448 63648758 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:40:55 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:41:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:41:15 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:41:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:41:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:41:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:41:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:41:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:41:15 [DEBUG   ] Finished updating board flags in 2.674 seconds
2018-11-26 17:41:35 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:41:55 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 17:42:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:42:14 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 17:42:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:42:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:42:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:42:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:42:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:42:14 [DEBUG   ] Finished updating board flags in 1.714 seconds
2018-11-26 17:42:15 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 17:42:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 17:42:45 [DEBUG   ] Got command RPT from MCS: ref #635205
2018-11-26 17:42:45 [DEBUG   ] summary = ERROR
2018-11-26 17:42:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635205  13 58448 63765805 A  ERRORERROR'
2018-11-26 17:42:46 [DEBUG   ] Got command RPT from MCS: ref #635206
2018-11-26 17:42:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:42:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635206  77 58448 63766009 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:42:46 [DEBUG   ] Got command RPT from MCS: ref #635207
2018-11-26 17:42:46 [DEBUG   ] summary = ERROR
2018-11-26 17:42:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635207  13 58448 63766201 A  ERRORERROR'
2018-11-26 17:42:46 [DEBUG   ] Got command RPT from MCS: ref #635209
2018-11-26 17:42:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:42:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635209  77 58448 63766405 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:42:46 [DEBUG   ] Got command RPT from MCS: ref #635210
2018-11-26 17:42:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:42:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635210  12 58448 63766609 A  ERRORB—™š'
2018-11-26 17:42:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 17:43:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa16, chip 1
2018-11-26 17:43:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:43:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:43:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:43:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:43:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:43:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:43:14 [DEBUG   ] Finished updating board flags in 1.627 seconds
2018-11-26 17:43:15 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 17:43:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 17:43:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 17:44:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:44:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:44:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:44:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:44:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:44:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:44:14 [DEBUG   ] Finished updating board flags in 1.561 seconds
2018-11-26 17:44:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 17:44:35 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 17:44:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 17:45:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:45:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:45:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:45:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:45:14 [WARNING ] (13, 1, 'bot')
2018-11-26 17:45:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:45:14 [DEBUG   ] Finished updating board flags in 1.565 seconds
2018-11-26 17:45:15 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 17:45:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 17:45:48 [DEBUG   ] Got command RPT from MCS: ref #635219
2018-11-26 17:45:48 [DEBUG   ] summary = ERROR
2018-11-26 17:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635219  13 58448 63948757 A  ERRORERROR'
2018-11-26 17:45:48 [DEBUG   ] Got command RPT from MCS: ref #635220
2018-11-26 17:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635220  77 58448 63948758 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:45:55 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 17:46:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:46:15 [DEBUG   ] Finished updating ant stats in 0.846 seconds
2018-11-26 17:46:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:46:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:46:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:46:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:46:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:46:15 [DEBUG   ] Finished updating board flags in 2.281 seconds
2018-11-26 17:46:35 [DEBUG   ] Finished updating ant stats in 0.757 seconds
2018-11-26 17:46:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 17:47:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:47:15 [DEBUG   ] Finished updating ant stats in 0.647 seconds
2018-11-26 17:47:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:47:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:47:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:47:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:47:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:47:15 [DEBUG   ] Finished updating board flags in 2.152 seconds
2018-11-26 17:47:35 [DEBUG   ] Finished updating ant stats in 0.558 seconds
2018-11-26 17:47:45 [DEBUG   ] Got command RPT from MCS: ref #635240
2018-11-26 17:47:45 [DEBUG   ] summary = ERROR
2018-11-26 17:47:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635240  13 58448 64065881 A  ERRORERROR'
2018-11-26 17:47:46 [DEBUG   ] Got command RPT from MCS: ref #635241
2018-11-26 17:47:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:47:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635241  77 58448 64066085 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:47:46 [DEBUG   ] Got command RPT from MCS: ref #635242
2018-11-26 17:47:46 [DEBUG   ] summary = ERROR
2018-11-26 17:47:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635242  13 58448 64066289 A  ERRORERROR'
2018-11-26 17:47:46 [DEBUG   ] Got command RPT from MCS: ref #635243
2018-11-26 17:47:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:47:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635243  77 58448 64066493 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:47:46 [DEBUG   ] Got command RPT from MCS: ref #635244
2018-11-26 17:47:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:47:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635244  12 58448 64066685 A  ERRORB–™š'
2018-11-26 17:47:56 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 17:48:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:48:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:48:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:48:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:48:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:48:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:48:15 [DEBUG   ] Finished updating board flags in 2.030 seconds
2018-11-26 17:48:16 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 17:48:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:48:56 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 17:49:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:49:16 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 17:49:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 17:49:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:49:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:49:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:49:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:49:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:49:16 [DEBUG   ] Finished updating board flags in 3.092 seconds
2018-11-26 17:49:36 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 17:49:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:50:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:50:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:50:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:50:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:50:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:50:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:50:15 [DEBUG   ] Finished updating board flags in 1.888 seconds
2018-11-26 17:50:16 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:50:36 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2018-11-26 17:50:48 [DEBUG   ] Got command RPT from MCS: ref #635249
2018-11-26 17:50:48 [DEBUG   ] summary = ERROR
2018-11-26 17:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635249  13 58448 64248757 A  ERRORERROR'
2018-11-26 17:50:48 [DEBUG   ] Got command RPT from MCS: ref #635250
2018-11-26 17:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635250  77 58448 64248758 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:50:56 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 17:51:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:51:16 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 17:51:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:51:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:51:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:51:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:51:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:51:16 [DEBUG   ] Finished updating board flags in 2.523 seconds
2018-11-26 17:51:36 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2018-11-26 17:51:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 17:52:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:52:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 17:52:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:52:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:52:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:52:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:52:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:52:15 [DEBUG   ] Finished updating board flags in 1.648 seconds
2018-11-26 17:52:16 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 17:52:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 17:52:45 [DEBUG   ] Got command RPT from MCS: ref #635272
2018-11-26 17:52:45 [DEBUG   ] summary = ERROR
2018-11-26 17:52:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635272  13 58448 64365969 A  ERRORERROR'
2018-11-26 17:52:46 [DEBUG   ] Got command RPT from MCS: ref #635273
2018-11-26 17:52:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:52:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635273  77 58448 64366161 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:52:46 [DEBUG   ] Got command RPT from MCS: ref #635274
2018-11-26 17:52:46 [DEBUG   ] summary = ERROR
2018-11-26 17:52:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635274  13 58448 64366365 A  ERRORERROR'
2018-11-26 17:52:46 [DEBUG   ] Got command RPT from MCS: ref #635276
2018-11-26 17:52:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:52:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635276  77 58448 64366569 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:52:46 [DEBUG   ] Got command RPT from MCS: ref #635277
2018-11-26 17:52:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:52:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635277  12 58448 64366773 A  ERRORB•™š'
2018-11-26 17:52:57 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 17:53:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:53:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:53:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:53:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:53:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:53:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:53:15 [DEBUG   ] Finished updating board flags in 1.622 seconds
2018-11-26 17:53:17 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 17:53:37 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 17:53:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:54:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:54:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:54:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:54:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:54:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:54:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:54:15 [DEBUG   ] Finished updating board flags in 1.574 seconds
2018-11-26 17:54:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 17:54:37 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 17:54:57 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 17:55:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:55:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:55:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:55:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:55:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:55:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:55:15 [DEBUG   ] Finished updating board flags in 1.418 seconds
2018-11-26 17:55:17 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 17:55:37 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 17:55:48 [DEBUG   ] Got command RPT from MCS: ref #635286
2018-11-26 17:55:48 [DEBUG   ] summary = ERROR
2018-11-26 17:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635286  13 58448 64548765 A  ERRORERROR'
2018-11-26 17:55:48 [DEBUG   ] Got command RPT from MCS: ref #635287
2018-11-26 17:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635287  77 58448 64548766 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:55:57 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 17:56:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:56:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:56:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:56:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:56:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:56:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:56:15 [DEBUG   ] Finished updating board flags in 1.416 seconds
2018-11-26 17:56:17 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 17:56:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 17:56:57 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 17:57:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:57:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:57:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:57:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:57:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:57:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:57:15 [DEBUG   ] Finished updating board flags in 1.418 seconds
2018-11-26 17:57:17 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 17:57:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 17:57:46 [DEBUG   ] Got command RPT from MCS: ref #635303
2018-11-26 17:57:46 [DEBUG   ] summary = ERROR
2018-11-26 17:57:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635303  13 58448 64666049 A  ERRORERROR'
2018-11-26 17:57:46 [DEBUG   ] Got command RPT from MCS: ref #635304
2018-11-26 17:57:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:57:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635304  77 58448 64666241 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:57:46 [DEBUG   ] Got command RPT from MCS: ref #635305
2018-11-26 17:57:46 [DEBUG   ] summary = ERROR
2018-11-26 17:57:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635305  13 58448 64666445 A  ERRORERROR'
2018-11-26 17:57:46 [DEBUG   ] Got command RPT from MCS: ref #635306
2018-11-26 17:57:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 17:57:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635306  77 58448 64666649 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 17:57:46 [DEBUG   ] Got command RPT from MCS: ref #635307
2018-11-26 17:57:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 17:57:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635307  12 58448 64666853 A  ERRORB–™š'
2018-11-26 17:57:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 17:58:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:58:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:58:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:58:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:58:15 [WARNING ] (13, 1, 'bot')
2018-11-26 17:58:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:58:15 [DEBUG   ] Finished updating board flags in 1.464 seconds
2018-11-26 17:58:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 17:58:37 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 17:58:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 17:59:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 17:59:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:59:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:59:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:59:16 [WARNING ] (13, 1, 'bot')
2018-11-26 17:59:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 17:59:16 [DEBUG   ] Finished updating board flags in 2.304 seconds
2018-11-26 17:59:17 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2018-11-26 17:59:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 17:59:57 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 18:00:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:00:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:00:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:00:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:00:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:00:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:00:16 [DEBUG   ] Finished updating board flags in 2.105 seconds
2018-11-26 18:00:17 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 18:00:37 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-26 18:00:48 [DEBUG   ] Got command RPT from MCS: ref #635312
2018-11-26 18:00:48 [DEBUG   ] summary = ERROR
2018-11-26 18:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635312  13 58448 64848769 A  ERRORERROR'
2018-11-26 18:00:48 [DEBUG   ] Got command RPT from MCS: ref #635313
2018-11-26 18:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635313  77 58448 64848770 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:00:57 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 18:01:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:01:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 18:01:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:01:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:01:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:01:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:01:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:01:16 [DEBUG   ] Finished updating board flags in 2.020 seconds
2018-11-26 18:01:18 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 18:01:38 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 18:01:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:02:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:02:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 18:02:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:02:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:02:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:02:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:02:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:02:16 [DEBUG   ] Finished updating board flags in 1.973 seconds
2018-11-26 18:02:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:02:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:02:46 [DEBUG   ] Got command RPT from MCS: ref #635335
2018-11-26 18:02:46 [DEBUG   ] summary = ERROR
2018-11-26 18:02:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635335  13 58448 64966125 A  ERRORERROR'
2018-11-26 18:02:46 [DEBUG   ] Got command RPT from MCS: ref #635336
2018-11-26 18:02:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:02:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635336  77 58448 64966329 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:02:46 [DEBUG   ] Got command RPT from MCS: ref #635337
2018-11-26 18:02:46 [DEBUG   ] summary = ERROR
2018-11-26 18:02:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635337  13 58448 64966521 A  ERRORERROR'
2018-11-26 18:02:46 [DEBUG   ] Got command RPT from MCS: ref #635338
2018-11-26 18:02:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:02:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635338  77 58448 64966725 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:02:46 [DEBUG   ] Got command RPT from MCS: ref #635340
2018-11-26 18:02:46 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:02:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635340  12 58448 64966929 A  ERRORB–™š'
2018-11-26 18:02:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:03:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 18:03:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:03:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:03:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:03:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:03:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:03:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:03:16 [DEBUG   ] Finished updating board flags in 1.833 seconds
2018-11-26 18:03:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:03:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:03:58 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:04:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:04:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:04:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:04:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:04:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:04:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:04:16 [DEBUG   ] Finished updating board flags in 1.813 seconds
2018-11-26 18:04:18 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:04:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:04:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:05:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:05:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:05:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:05:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:05:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:05:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:05:16 [DEBUG   ] Finished updating board flags in 1.807 seconds
2018-11-26 18:05:18 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:05:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:05:48 [DEBUG   ] Got command RPT from MCS: ref #635349
2018-11-26 18:05:48 [DEBUG   ] summary = ERROR
2018-11-26 18:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635349  13 58448 65148769 A  ERRORERROR'
2018-11-26 18:05:48 [DEBUG   ] Got command RPT from MCS: ref #635350
2018-11-26 18:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635350  77 58448 65148770 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:05:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:06:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:06:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:06:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:06:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:06:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:06:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:06:16 [DEBUG   ] Finished updating board flags in 1.800 seconds
2018-11-26 18:06:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:06:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:06:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:07:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:07:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:07:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:07:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:07:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:07:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:07:16 [DEBUG   ] Finished updating board flags in 1.635 seconds
2018-11-26 18:07:18 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 18:07:38 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 18:07:46 [DEBUG   ] Got command RPT from MCS: ref #635370
2018-11-26 18:07:46 [DEBUG   ] summary = ERROR
2018-11-26 18:07:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635370  13 58448 65266209 A  ERRORERROR'
2018-11-26 18:07:46 [DEBUG   ] Got command RPT from MCS: ref #635371
2018-11-26 18:07:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:07:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635371  77 58448 65266413 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:07:46 [DEBUG   ] Got command RPT from MCS: ref #635372
2018-11-26 18:07:46 [DEBUG   ] summary = ERROR
2018-11-26 18:07:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635372  13 58448 65266606 A  ERRORERROR'
2018-11-26 18:07:46 [DEBUG   ] Got command RPT from MCS: ref #635373
2018-11-26 18:07:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:07:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635373  77 58448 65266809 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:07:47 [DEBUG   ] Got command RPT from MCS: ref #635374
2018-11-26 18:07:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:07:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635374  12 58448 65267013 A  ERRORB•™š'
2018-11-26 18:07:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:08:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:08:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:08:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:08:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:08:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:08:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:08:16 [DEBUG   ] Finished updating board flags in 1.519 seconds
2018-11-26 18:08:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:08:38 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 18:08:58 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 18:09:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:09:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:09:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:09:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:09:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:09:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:09:16 [DEBUG   ] Finished updating board flags in 1.501 seconds
2018-11-26 18:09:18 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 18:09:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:09:58 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 18:10:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:10:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:10:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:10:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:10:16 [WARNING ] (13, 1, 'bot')
2018-11-26 18:10:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:10:16 [DEBUG   ] Finished updating board flags in 1.526 seconds
2018-11-26 18:10:18 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2018-11-26 18:10:38 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2018-11-26 18:10:48 [DEBUG   ] Got command RPT from MCS: ref #635379
2018-11-26 18:10:48 [DEBUG   ] summary = ERROR
2018-11-26 18:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635379  13 58448 65448761 A  ERRORERROR'
2018-11-26 18:10:48 [DEBUG   ] Got command RPT from MCS: ref #635380
2018-11-26 18:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635380  77 58448 65448762 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:10:58 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:11:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-26 18:11:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:11:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:11:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:11:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:11:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:11:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:11:17 [DEBUG   ] Finished updating board flags in 2.236 seconds
2018-11-26 18:11:18 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 18:11:38 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 18:11:58 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 18:12:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:12:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:12:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:12:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:12:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:12:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:12:17 [DEBUG   ] Finished updating board flags in 2.161 seconds
2018-11-26 18:12:18 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:12:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:12:46 [DEBUG   ] Got command RPT from MCS: ref #635398
2018-11-26 18:12:46 [DEBUG   ] summary = ERROR
2018-11-26 18:12:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635398  13 58448 65566289 A  ERRORERROR'
2018-11-26 18:12:46 [DEBUG   ] Got command RPT from MCS: ref #635399
2018-11-26 18:12:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:12:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635399  77 58448 65566493 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:12:46 [DEBUG   ] Got command RPT from MCS: ref #635400
2018-11-26 18:12:46 [DEBUG   ] summary = ERROR
2018-11-26 18:12:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635400  13 58448 65566685 A  ERRORERROR'
2018-11-26 18:12:46 [DEBUG   ] Got command RPT from MCS: ref #635402
2018-11-26 18:12:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:12:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635402  77 58448 65566889 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:12:47 [DEBUG   ] Got command RPT from MCS: ref #635404
2018-11-26 18:12:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:12:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635404  12 58448 65567093 A  ERRORB—™š'
2018-11-26 18:12:58 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 18:13:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:13:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:13:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:13:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:13:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:13:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:13:17 [DEBUG   ] Finished updating board flags in 1.997 seconds
2018-11-26 18:13:18 [DEBUG   ] Finished updating ant stats in 0.594 seconds
2018-11-26 18:13:38 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-26 18:13:59 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 18:14:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:14:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 18:14:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:14:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:14:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:14:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:14:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:14:17 [DEBUG   ] Finished updating board flags in 1.801 seconds
2018-11-26 18:14:19 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 18:14:39 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:14:59 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:15:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:15:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:15:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:15:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:15:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:15:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:15:17 [DEBUG   ] Finished updating board flags in 1.807 seconds
2018-11-26 18:15:19 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:15:39 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:15:48 [DEBUG   ] Got command RPT from MCS: ref #635412
2018-11-26 18:15:48 [DEBUG   ] summary = ERROR
2018-11-26 18:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635412  13 58448 65748769 A  ERRORERROR'
2018-11-26 18:15:48 [DEBUG   ] Got command RPT from MCS: ref #635413
2018-11-26 18:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635413  77 58448 65748770 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:15:59 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:16:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 18:16:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:16:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:16:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:16:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:16:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:16:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:16:17 [DEBUG   ] Finished updating board flags in 1.856 seconds
2018-11-26 18:16:19 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 18:16:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:16:59 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:17:11 [DEBUG   ] Got command TBW from MCS: ref #635434
2018-11-26 18:17:11 [DEBUG   ] tbw = exited with status True, code 0x00
2018-11-26 18:17:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBW   635434   8 58448 65831374 A  ERROR'
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 pkill tbn
2018-11-26 18:17:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 pkill tbn
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa27', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '1', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa24', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '2', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa23', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '3', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa18', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '4', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa10', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '5', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa32', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '6', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa11', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '7', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa30', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '8', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa12', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '9', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa8', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '10', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa13', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '11', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa26', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '12', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa14', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '13', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa22', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '14', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa20', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '15', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa1', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '16', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa5', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '17', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa25', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '18', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa35', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '19', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa19', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '20', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa21', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '21', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa6', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '22', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa7', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '23', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa33', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '24', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa15', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '25', '12000000']
2018-11-26 18:17:14 [DEBUG   ] spawn_process: at 2018-11-26 18:17:14 running ['/usr/local/bin/lwa28', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '26', '12000000']
2018-11-26 18:17:14 [INFO    ] Finished spawning TBW
2018-11-26 18:17:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:17:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:17:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:17:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:17:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:17:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:17:17 [DEBUG   ] Finished updating board flags in 1.553 seconds
2018-11-26 18:17:46 [DEBUG   ] Got command RPT from MCS: ref #635437
2018-11-26 18:17:46 [DEBUG   ] summary = ERROR
2018-11-26 18:17:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635437  13 58448 65866361 A  ERRORERROR'
2018-11-26 18:17:46 [DEBUG   ] Got command RPT from MCS: ref #635438
2018-11-26 18:17:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:17:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635438  77 58448 65866565 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:17:46 [DEBUG   ] Got command RPT from MCS: ref #635439
2018-11-26 18:17:46 [DEBUG   ] summary = ERROR
2018-11-26 18:17:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635439  13 58448 65866769 A  ERRORERROR'
2018-11-26 18:17:46 [DEBUG   ] Got command RPT from MCS: ref #635440
2018-11-26 18:17:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:17:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635440  77 58448 65866973 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:17:47 [DEBUG   ] Got command RPT from MCS: ref #635441
2018-11-26 18:17:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:17:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635441  12 58448 65867165 A  ERRORB—™š'
2018-11-26 18:18:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:18:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:18:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:18:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:18:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:18:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:18:17 [DEBUG   ] Finished updating board flags in 1.550 seconds
2018-11-26 18:19:02 [INFO    ] __waitForTBW - TBW finished in 108.006 s
2018-11-26 18:19:03 [DEBUG   ] Finished updating ant stats in 0.377 seconds
2018-11-26 18:19:03 [INFO    ] Started antennas statistics background thread
2018-11-26 18:19:03 [INFO    ] Resuming TBN at 38.100 MHz after TBW acquisition
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '1', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '2', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '3', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '4', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '5', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '6', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '7', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '8', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '9', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '10', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '11', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '12', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '13', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '14', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '15', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '16', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '17', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '18', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '19', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '20', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '21', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '22', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '23', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '24', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '25', '1543256345']
2018-11-26 18:19:03 [DEBUG   ] spawn_process: at 2018-11-26 18:19:03 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '26', '1543256345']
2018-11-26 18:19:03 [INFO    ] Finished spawning TBN in 0.050 s
2018-11-26 18:19:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:19:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:19:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:19:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:19:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:19:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:19:17 [DEBUG   ] Finished updating board flags in 1.635 seconds
2018-11-26 18:19:24 [DEBUG   ] Finished updating ant stats in 1.270 seconds
2018-11-26 18:19:44 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:20:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:20:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:20:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:20:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:20:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:20:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:20:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:20:17 [DEBUG   ] Finished updating board flags in 1.635 seconds
2018-11-26 18:20:24 [DEBUG   ] Finished updating ant stats in 1.193 seconds
2018-11-26 18:20:44 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 18:20:48 [DEBUG   ] Got command RPT from MCS: ref #635451
2018-11-26 18:20:48 [DEBUG   ] summary = ERROR
2018-11-26 18:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635451  13 58448 66048769 A  ERRORERROR'
2018-11-26 18:20:48 [DEBUG   ] Got command RPT from MCS: ref #635452
2018-11-26 18:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635452  77 58448 66048770 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:21:04 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 18:21:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:21:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:21:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:21:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:21:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:21:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:21:17 [DEBUG   ] Finished updating board flags in 1.635 seconds
2018-11-26 18:21:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:21:44 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 18:22:04 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:22:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:22:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:22:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:22:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:22:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:22:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:22:17 [DEBUG   ] Finished updating board flags in 1.631 seconds
2018-11-26 18:22:24 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 18:22:44 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:22:46 [DEBUG   ] Got command RPT from MCS: ref #635474
2018-11-26 18:22:46 [DEBUG   ] summary = ERROR
2018-11-26 18:22:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635474  13 58448 66166453 A  ERRORERROR'
2018-11-26 18:22:46 [DEBUG   ] Got command RPT from MCS: ref #635475
2018-11-26 18:22:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:22:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635475  77 58448 66166645 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:22:46 [DEBUG   ] Got command RPT from MCS: ref #635476
2018-11-26 18:22:46 [DEBUG   ] summary = ERROR
2018-11-26 18:22:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635476  13 58448 66166849 A  ERRORERROR'
2018-11-26 18:22:47 [DEBUG   ] Got command RPT from MCS: ref #635478
2018-11-26 18:22:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:22:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635478  77 58448 66167053 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:22:47 [DEBUG   ] Got command RPT from MCS: ref #635480
2018-11-26 18:22:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:22:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635480  12 58448 66167245 A  ERRORB–™š'
2018-11-26 18:23:04 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 18:23:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:23:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:23:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:23:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:23:17 [WARNING ] (13, 1, 'bot')
2018-11-26 18:23:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:23:17 [DEBUG   ] Finished updating board flags in 1.655 seconds
2018-11-26 18:23:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:23:44 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 18:24:04 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:24:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:24:17 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 18:24:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:24:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:24:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:24:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:24:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:24:18 [DEBUG   ] Finished updating board flags in 1.691 seconds
2018-11-26 18:24:24 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 18:24:44 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 18:25:04 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 18:25:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:25:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:25:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:25:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:25:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:25:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:25:18 [DEBUG   ] Finished updating board flags in 2.322 seconds
2018-11-26 18:25:24 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 18:25:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 18:25:48 [DEBUG   ] Got command RPT from MCS: ref #635488
2018-11-26 18:25:48 [DEBUG   ] summary = ERROR
2018-11-26 18:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635488  13 58448 66348773 A  ERRORERROR'
2018-11-26 18:25:48 [DEBUG   ] Got command RPT from MCS: ref #635489
2018-11-26 18:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635489  77 58448 66348774 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:26:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:26:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:26:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:26:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:26:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:26:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:26:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:26:18 [DEBUG   ] Finished updating board flags in 2.055 seconds
2018-11-26 18:26:24 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 18:26:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:27:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 18:27:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:27:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:27:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:27:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:27:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:27:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:27:18 [DEBUG   ] Finished updating board flags in 2.013 seconds
2018-11-26 18:27:24 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 18:27:44 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 18:27:46 [DEBUG   ] Got command RPT from MCS: ref #635505
2018-11-26 18:27:46 [DEBUG   ] summary = ERROR
2018-11-26 18:27:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635505  13 58448 66466525 A  ERRORERROR'
2018-11-26 18:27:46 [DEBUG   ] Got command RPT from MCS: ref #635506
2018-11-26 18:27:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:27:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635506  77 58448 66466729 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:27:46 [DEBUG   ] Got command RPT from MCS: ref #635507
2018-11-26 18:27:46 [DEBUG   ] summary = ERROR
2018-11-26 18:27:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635507  13 58448 66466933 A  ERRORERROR'
2018-11-26 18:27:47 [DEBUG   ] Got command RPT from MCS: ref #635508
2018-11-26 18:27:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:27:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635508  77 58448 66467137 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:27:47 [DEBUG   ] Got command RPT from MCS: ref #635509
2018-11-26 18:27:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:27:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635509  12 58448 66467329 A  ERRORB”™š'
2018-11-26 18:28:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:28:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:28:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:28:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:28:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:28:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:28:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:28:18 [DEBUG   ] Finished updating board flags in 1.991 seconds
2018-11-26 18:28:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 18:28:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:29:04 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 18:29:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:29:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:29:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:29:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:29:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:29:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:29:18 [DEBUG   ] Finished updating board flags in 1.816 seconds
2018-11-26 18:29:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:29:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:30:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 18:30:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 18:30:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:30:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:30:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:30:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:30:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:30:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:30:18 [DEBUG   ] Finished updating board flags in 1.855 seconds
2018-11-26 18:30:24 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 18:30:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:30:48 [DEBUG   ] Got command RPT from MCS: ref #635514
2018-11-26 18:30:48 [DEBUG   ] summary = ERROR
2018-11-26 18:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635514  13 58448 66648773 A  ERRORERROR'
2018-11-26 18:30:48 [DEBUG   ] Got command RPT from MCS: ref #635515
2018-11-26 18:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635515  77 58448 66648774 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:31:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 18:31:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:31:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:31:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:31:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:31:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:31:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:31:18 [DEBUG   ] Finished updating board flags in 1.850 seconds
2018-11-26 18:31:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:31:44 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 18:32:04 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 18:32:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:32:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:32:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:32:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:32:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:32:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:32:18 [DEBUG   ] Finished updating board flags in 1.808 seconds
2018-11-26 18:32:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 18:32:44 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 18:32:46 [DEBUG   ] Got command RPT from MCS: ref #635537
2018-11-26 18:32:46 [DEBUG   ] summary = ERROR
2018-11-26 18:32:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635537  13 58448 66766613 A  ERRORERROR'
2018-11-26 18:32:46 [DEBUG   ] Got command RPT from MCS: ref #635538
2018-11-26 18:32:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:32:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635538  77 58448 66766805 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:32:47 [DEBUG   ] Got command RPT from MCS: ref #635539
2018-11-26 18:32:47 [DEBUG   ] summary = ERROR
2018-11-26 18:32:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635539  13 58448 66767009 A  ERRORERROR'
2018-11-26 18:32:47 [DEBUG   ] Got command RPT from MCS: ref #635541
2018-11-26 18:32:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:32:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635541  77 58448 66767213 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:32:47 [DEBUG   ] Got command RPT from MCS: ref #635542
2018-11-26 18:32:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:32:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635542  12 58448 66767417 A  ERRORB–™š'
2018-11-26 18:33:04 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 18:33:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:33:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:33:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:33:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:33:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:33:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:33:18 [DEBUG   ] Finished updating board flags in 1.688 seconds
2018-11-26 18:33:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:33:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 18:34:04 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 18:34:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:34:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 18:34:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:34:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:34:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:34:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:34:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:34:18 [DEBUG   ] Finished updating board flags in 1.624 seconds
2018-11-26 18:34:24 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 18:34:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:35:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 18:35:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:35:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:35:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:35:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:35:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:35:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:35:18 [DEBUG   ] Finished updating board flags in 1.615 seconds
2018-11-26 18:35:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 18:35:44 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 18:35:48 [DEBUG   ] Got command RPT from MCS: ref #635551
2018-11-26 18:35:48 [DEBUG   ] summary = ERROR
2018-11-26 18:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635551  13 58448 66948769 A  ERRORERROR'
2018-11-26 18:35:48 [DEBUG   ] Got command RPT from MCS: ref #635552
2018-11-26 18:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635552  77 58448 66948770 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:36:04 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 18:36:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:36:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 18:36:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:36:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:36:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:36:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:36:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:36:18 [DEBUG   ] Finished updating board flags in 1.579 seconds
2018-11-26 18:36:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 18:36:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:37:04 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 18:37:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:37:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:37:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:37:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:37:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:37:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:37:18 [DEBUG   ] Finished updating board flags in 1.401 seconds
2018-11-26 18:37:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:37:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 18:37:46 [DEBUG   ] Got command RPT from MCS: ref #635572
2018-11-26 18:37:46 [DEBUG   ] summary = ERROR
2018-11-26 18:37:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635572  13 58448 67066689 A  ERRORERROR'
2018-11-26 18:37:46 [DEBUG   ] Got command RPT from MCS: ref #635573
2018-11-26 18:37:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:37:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635573  77 58448 67066893 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:37:47 [DEBUG   ] Got command RPT from MCS: ref #635574
2018-11-26 18:37:47 [DEBUG   ] summary = ERROR
2018-11-26 18:37:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635574  13 58448 67067097 A  ERRORERROR'
2018-11-26 18:37:47 [DEBUG   ] Got command RPT from MCS: ref #635575
2018-11-26 18:37:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:37:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635575  77 58448 67067289 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:37:47 [DEBUG   ] Got command RPT from MCS: ref #635576
2018-11-26 18:37:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:37:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635576  12 58448 67067493 A  ERRORB—™š'
2018-11-26 18:38:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:38:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 18:38:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:38:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:38:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:38:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:38:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:38:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:38:18 [DEBUG   ] Finished updating board flags in 1.563 seconds
2018-11-26 18:38:24 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 18:38:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:39:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 18:39:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:39:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:39:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:39:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:39:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:39:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:39:19 [DEBUG   ] Finished updating board flags in 1.602 seconds
2018-11-26 18:39:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:39:44 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 18:40:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:40:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:40:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:40:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:40:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:40:18 [WARNING ] (13, 1, 'bot')
2018-11-26 18:40:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:40:18 [DEBUG   ] Finished updating board flags in 1.346 seconds
2018-11-26 18:40:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 18:40:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:40:48 [DEBUG   ] Got command RPT from MCS: ref #635581
2018-11-26 18:40:48 [DEBUG   ] summary = ERROR
2018-11-26 18:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635581  13 58448 67248777 A  ERRORERROR'
2018-11-26 18:40:48 [DEBUG   ] Got command RPT from MCS: ref #635582
2018-11-26 18:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635582  77 58448 67248778 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:41:04 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 18:41:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:41:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:41:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:41:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:41:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:41:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:41:19 [DEBUG   ] Finished updating board flags in 2.097 seconds
2018-11-26 18:41:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:41:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 18:42:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 18:42:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:42:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:42:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:42:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:42:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:42:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:42:19 [DEBUG   ] Finished updating board flags in 1.982 seconds
2018-11-26 18:42:24 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 18:42:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:42:46 [DEBUG   ] Got command RPT from MCS: ref #635600
2018-11-26 18:42:46 [DEBUG   ] summary = ERROR
2018-11-26 18:42:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635600  13 58448 67366769 A  ERRORERROR'
2018-11-26 18:42:46 [DEBUG   ] Got command RPT from MCS: ref #635601
2018-11-26 18:42:46 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:42:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635601  77 58448 67366973 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:42:47 [DEBUG   ] Got command RPT from MCS: ref #635602
2018-11-26 18:42:47 [DEBUG   ] summary = ERROR
2018-11-26 18:42:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635602  13 58448 67367177 A  ERRORERROR'
2018-11-26 18:42:47 [DEBUG   ] Got command RPT from MCS: ref #635604
2018-11-26 18:42:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:42:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635604  77 58448 67367381 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:42:47 [DEBUG   ] Got command RPT from MCS: ref #635605
2018-11-26 18:42:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:42:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635605  12 58448 67367573 A  ERRORB•™š'
2018-11-26 18:43:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 18:43:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:43:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:43:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:43:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:43:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:43:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:43:19 [DEBUG   ] Finished updating board flags in 1.817 seconds
2018-11-26 18:43:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:43:44 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 18:44:04 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 18:44:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:44:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:44:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:44:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:44:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:44:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:44:19 [DEBUG   ] Finished updating board flags in 1.898 seconds
2018-11-26 18:44:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:44:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 18:45:04 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 18:45:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 18:45:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:45:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:45:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:45:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:45:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:45:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:45:19 [DEBUG   ] Finished updating board flags in 1.694 seconds
2018-11-26 18:45:24 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 18:45:44 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 18:45:48 [DEBUG   ] Got command RPT from MCS: ref #635614
2018-11-26 18:45:48 [DEBUG   ] summary = ERROR
2018-11-26 18:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635614  13 58448 67548777 A  ERRORERROR'
2018-11-26 18:45:48 [DEBUG   ] Got command RPT from MCS: ref #635615
2018-11-26 18:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635615  77 58448 67548778 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:46:04 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 18:46:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:46:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:46:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:46:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:46:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:46:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:46:19 [DEBUG   ] Finished updating board flags in 1.622 seconds
2018-11-26 18:46:25 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 18:46:45 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 18:47:05 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:47:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 18:47:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:47:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:47:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:47:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:47:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:47:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:47:19 [DEBUG   ] Finished updating board flags in 1.582 seconds
2018-11-26 18:47:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:47:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:47:46 [DEBUG   ] Got command RPT from MCS: ref #635635
2018-11-26 18:47:46 [DEBUG   ] summary = ERROR
2018-11-26 18:47:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635635  13 58448 67666857 A  ERRORERROR'
2018-11-26 18:47:47 [DEBUG   ] Got command RPT from MCS: ref #635636
2018-11-26 18:47:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:47:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635636  77 58448 67667061 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:47:47 [DEBUG   ] Got command RPT from MCS: ref #635637
2018-11-26 18:47:47 [DEBUG   ] summary = ERROR
2018-11-26 18:47:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635637  13 58448 67667265 A  ERRORERROR'
2018-11-26 18:47:47 [DEBUG   ] Got command RPT from MCS: ref #635638
2018-11-26 18:47:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:47:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635638  77 58448 67667469 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:47:47 [DEBUG   ] Got command RPT from MCS: ref #635639
2018-11-26 18:47:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:47:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635639  12 58448 67667661 A  ERRORB–™š'
2018-11-26 18:48:05 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:48:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:48:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:48:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:48:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:48:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:48:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:48:19 [DEBUG   ] Finished updating board flags in 1.428 seconds
2018-11-26 18:48:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:48:45 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:49:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:49:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-26 18:49:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:49:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:49:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:49:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:49:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:49:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:49:19 [DEBUG   ] Finished updating board flags in 1.412 seconds
2018-11-26 18:49:25 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:49:45 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:50:05 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 18:50:18 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 18:50:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:50:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 18:50:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:50:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:50:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:50:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:50:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:50:19 [DEBUG   ] Finished updating board flags in 1.436 seconds
2018-11-26 18:50:25 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:50:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:50:48 [DEBUG   ] Got command RPT from MCS: ref #635644
2018-11-26 18:50:48 [DEBUG   ] summary = ERROR
2018-11-26 18:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635644  13 58448 67848769 A  ERRORERROR'
2018-11-26 18:50:48 [DEBUG   ] Got command RPT from MCS: ref #635645
2018-11-26 18:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635645  77 58448 67848770 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:51:05 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:51:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 18:51:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:51:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:51:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:51:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:51:19 [WARNING ] (13, 1, 'bot')
2018-11-26 18:51:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:51:19 [DEBUG   ] Finished updating board flags in 1.579 seconds
2018-11-26 18:51:25 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 18:51:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:52:05 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:52:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa16, chip 1
2018-11-26 18:52:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:52:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 18:52:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:52:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:52:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:52:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:52:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:52:20 [DEBUG   ] Finished updating board flags in 2.270 seconds
2018-11-26 18:52:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:52:45 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:52:46 [DEBUG   ] Got command RPT from MCS: ref #635667
2018-11-26 18:52:46 [DEBUG   ] summary = ERROR
2018-11-26 18:52:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635667  13 58448 67966949 A  ERRORERROR'
2018-11-26 18:52:47 [DEBUG   ] Got command RPT from MCS: ref #635668
2018-11-26 18:52:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:52:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635668  77 58448 67967153 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:52:47 [DEBUG   ] Got command RPT from MCS: ref #635669
2018-11-26 18:52:47 [DEBUG   ] summary = ERROR
2018-11-26 18:52:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635669  13 58448 67967345 A  ERRORERROR'
2018-11-26 18:52:47 [DEBUG   ] Got command RPT from MCS: ref #635671
2018-11-26 18:52:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:52:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635671  77 58448 67967549 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:52:47 [DEBUG   ] Got command RPT from MCS: ref #635672
2018-11-26 18:52:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:52:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635672  12 58448 67967753 A  ERRORB—™š'
2018-11-26 18:53:05 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:53:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:53:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:53:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:53:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:53:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:53:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:53:20 [DEBUG   ] Finished updating board flags in 2.110 seconds
2018-11-26 18:53:25 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:53:45 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:54:05 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 18:54:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:54:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:54:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:54:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:54:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:54:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:54:20 [DEBUG   ] Finished updating board flags in 1.995 seconds
2018-11-26 18:54:25 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:54:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 18:55:05 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:55:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:55:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:55:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:55:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:55:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:55:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:55:20 [DEBUG   ] Finished updating board flags in 1.820 seconds
2018-11-26 18:55:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:55:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:55:48 [DEBUG   ] Got command RPT from MCS: ref #635681
2018-11-26 18:55:48 [DEBUG   ] summary = ERROR
2018-11-26 18:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635681  13 58448 68148773 A  ERRORERROR'
2018-11-26 18:55:48 [DEBUG   ] Got command RPT from MCS: ref #635682
2018-11-26 18:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635682  77 58448 68148774 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:56:05 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:56:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:56:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:56:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:56:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:56:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:56:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:56:20 [DEBUG   ] Finished updating board flags in 1.783 seconds
2018-11-26 18:56:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:56:45 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 18:57:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 18:57:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:57:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 18:57:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:57:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:57:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:57:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:57:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:57:20 [DEBUG   ] Finished updating board flags in 1.655 seconds
2018-11-26 18:57:25 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2018-11-26 18:57:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:57:47 [DEBUG   ] Got command RPT from MCS: ref #635698
2018-11-26 18:57:47 [DEBUG   ] summary = ERROR
2018-11-26 18:57:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635698  13 58448 68267037 A  ERRORERROR'
2018-11-26 18:57:47 [DEBUG   ] Got command RPT from MCS: ref #635699
2018-11-26 18:57:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:57:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635699  77 58448 68267241 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:57:47 [DEBUG   ] Got command RPT from MCS: ref #635700
2018-11-26 18:57:47 [DEBUG   ] summary = ERROR
2018-11-26 18:57:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635700  13 58448 68267433 A  ERRORERROR'
2018-11-26 18:57:47 [DEBUG   ] Got command RPT from MCS: ref #635701
2018-11-26 18:57:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 18:57:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635701  77 58448 68267637 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 18:57:47 [DEBUG   ] Got command RPT from MCS: ref #635702
2018-11-26 18:57:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 18:57:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635702  12 58448 68267833 A  ERRORB–™š'
2018-11-26 18:58:05 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 18:58:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:58:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 18:58:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:58:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:58:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:58:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:58:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:58:20 [DEBUG   ] Finished updating board flags in 1.538 seconds
2018-11-26 18:58:25 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 18:58:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:59:05 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 18:59:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 18:59:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:59:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:59:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:59:20 [WARNING ] (13, 1, 'bot')
2018-11-26 18:59:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 18:59:20 [DEBUG   ] Finished updating board flags in 1.551 seconds
2018-11-26 18:59:25 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 18:59:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:00:05 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:00:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:00:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:00:20 [WARNING ] (13, 1, 'bot')
2018-11-26 19:00:20 [WARNING ] (13, 1, 'bot')
2018-11-26 19:00:20 [WARNING ] (13, 1, 'bot')
2018-11-26 19:00:20 [WARNING ] (13, 1, 'bot')
2018-11-26 19:00:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:00:20 [DEBUG   ] Finished updating board flags in 1.558 seconds
2018-11-26 19:00:25 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:00:45 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 19:00:48 [DEBUG   ] Got command RPT from MCS: ref #635707
2018-11-26 19:00:48 [DEBUG   ] summary = ERROR
2018-11-26 19:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635707  13 58448 68448773 A  ERRORERROR'
2018-11-26 19:00:48 [DEBUG   ] Got command RPT from MCS: ref #635708
2018-11-26 19:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635708  77 58448 68448774 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:01:05 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:01:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:01:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:01:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:01:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:01:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:01:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:01:21 [DEBUG   ] Finished updating board flags in 2.189 seconds
2018-11-26 19:01:25 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:01:45 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:02:05 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:02:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:02:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:02:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:02:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:02:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:02:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:02:21 [DEBUG   ] Finished updating board flags in 1.940 seconds
2018-11-26 19:02:25 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:02:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:02:47 [DEBUG   ] Got command RPT from MCS: ref #635730
2018-11-26 19:02:47 [DEBUG   ] summary = ERROR
2018-11-26 19:02:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635730  13 58448 68567117 A  ERRORERROR'
2018-11-26 19:02:47 [DEBUG   ] Got command RPT from MCS: ref #635731
2018-11-26 19:02:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:02:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635731  77 58448 68567321 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:02:47 [DEBUG   ] Got command RPT from MCS: ref #635732
2018-11-26 19:02:47 [DEBUG   ] summary = ERROR
2018-11-26 19:02:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635732  13 58448 68567525 A  ERRORERROR'
2018-11-26 19:02:47 [DEBUG   ] Got command RPT from MCS: ref #635734
2018-11-26 19:02:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:02:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635734  77 58448 68567717 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:02:47 [DEBUG   ] Got command RPT from MCS: ref #635735
2018-11-26 19:02:47 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:02:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635735  12 58448 68567921 A  ERRORB•™š'
2018-11-26 19:03:05 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:03:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:03:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:03:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:03:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:03:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:03:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:03:21 [DEBUG   ] Finished updating board flags in 1.827 seconds
2018-11-26 19:03:25 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:03:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:04:05 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:04:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:04:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:04:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:04:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:04:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:04:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:04:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:04:21 [DEBUG   ] Finished updating board flags in 1.760 seconds
2018-11-26 19:04:25 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:04:45 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 19:05:05 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2018-11-26 19:05:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:05:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:05:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:05:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:05:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:05:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:05:21 [DEBUG   ] Finished updating board flags in 1.621 seconds
2018-11-26 19:05:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 19:05:45 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 19:05:48 [DEBUG   ] Got command RPT from MCS: ref #635744
2018-11-26 19:05:48 [DEBUG   ] summary = ERROR
2018-11-26 19:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635744  13 58448 68748773 A  ERRORERROR'
2018-11-26 19:05:48 [DEBUG   ] Got command RPT from MCS: ref #635745
2018-11-26 19:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635745  77 58448 68748774 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:06:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 19:06:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:06:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:06:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:06:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:06:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:06:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:06:21 [DEBUG   ] Finished updating board flags in 1.579 seconds
2018-11-26 19:06:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 19:06:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 19:07:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 19:07:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:07:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:07:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:07:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:07:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:07:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:07:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:07:21 [DEBUG   ] Finished updating board flags in 1.413 seconds
2018-11-26 19:07:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 19:07:45 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 19:07:47 [DEBUG   ] Got command RPT from MCS: ref #635765
2018-11-26 19:07:47 [DEBUG   ] summary = ERROR
2018-11-26 19:07:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635765  13 58448 68867213 A  ERRORERROR'
2018-11-26 19:07:47 [DEBUG   ] Got command RPT from MCS: ref #635766
2018-11-26 19:07:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:07:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635766  77 58448 68867405 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:07:47 [DEBUG   ] Got command RPT from MCS: ref #635767
2018-11-26 19:07:47 [DEBUG   ] summary = ERROR
2018-11-26 19:07:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635767  13 58448 68867609 A  ERRORERROR'
2018-11-26 19:07:47 [DEBUG   ] Got command RPT from MCS: ref #635768
2018-11-26 19:07:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:07:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635768  77 58448 68867813 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:07:48 [DEBUG   ] Got command RPT from MCS: ref #635769
2018-11-26 19:07:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635769  12 58448 68868005 A  ERRORB”™š'
2018-11-26 19:08:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 19:08:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:08:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:08:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:08:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:08:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:08:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:08:21 [DEBUG   ] Finished updating board flags in 1.542 seconds
2018-11-26 19:08:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 19:08:45 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 19:09:05 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2018-11-26 19:09:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa16, chip 1
2018-11-26 19:09:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:09:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:09:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:09:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:09:21 [WARNING ] (13, 1, 'bot')
2018-11-26 19:09:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:09:21 [DEBUG   ] Finished updating board flags in 1.575 seconds
2018-11-26 19:09:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 19:09:46 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2018-11-26 19:10:05 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:10:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:10:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:10:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:10:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:10:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:10:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:10:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:10:22 [DEBUG   ] Finished updating board flags in 2.190 seconds
2018-11-26 19:10:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:10:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:10:48 [DEBUG   ] Got command RPT from MCS: ref #635774
2018-11-26 19:10:48 [DEBUG   ] summary = ERROR
2018-11-26 19:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635774  13 58448 69048785 A  ERRORERROR'
2018-11-26 19:10:48 [DEBUG   ] Got command RPT from MCS: ref #635775
2018-11-26 19:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635775  77 58448 69048786 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:11:05 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:11:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:11:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:11:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:11:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:11:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:11:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:11:22 [DEBUG   ] Finished updating board flags in 2.019 seconds
2018-11-26 19:11:25 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:11:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 19:12:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:12:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:12:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:12:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:12:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:12:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:12:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:12:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:12:22 [DEBUG   ] Finished updating board flags in 1.885 seconds
2018-11-26 19:12:25 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:12:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:12:47 [DEBUG   ] Got command RPT from MCS: ref #635793
2018-11-26 19:12:47 [DEBUG   ] summary = ERROR
2018-11-26 19:12:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635793  13 58448 69167297 A  ERRORERROR'
2018-11-26 19:12:47 [DEBUG   ] Got command RPT from MCS: ref #635794
2018-11-26 19:12:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:12:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635794  77 58448 69167489 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:12:47 [DEBUG   ] Got command RPT from MCS: ref #635795
2018-11-26 19:12:47 [DEBUG   ] summary = ERROR
2018-11-26 19:12:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635795  13 58448 69167693 A  ERRORERROR'
2018-11-26 19:12:47 [DEBUG   ] Got command RPT from MCS: ref #635797
2018-11-26 19:12:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:12:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635797  77 58448 69167897 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:12:48 [DEBUG   ] Got command RPT from MCS: ref #635799
2018-11-26 19:12:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635799  12 58448 69168101 A  ERRORB—™š'
2018-11-26 19:13:05 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 19:13:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:13:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:13:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:13:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:13:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:13:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:13:22 [DEBUG   ] Finished updating board flags in 1.812 seconds
2018-11-26 19:13:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 19:13:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:14:05 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:14:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:14:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:14:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:14:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:14:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:14:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:14:22 [DEBUG   ] Finished updating board flags in 1.835 seconds
2018-11-26 19:14:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:14:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:15:05 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:15:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa16, chip 1
2018-11-26 19:15:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 19:15:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:15:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:15:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:15:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:15:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:15:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:15:22 [DEBUG   ] Finished updating board flags in 1.746 seconds
2018-11-26 19:15:25 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:15:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 19:15:48 [DEBUG   ] Got command RPT from MCS: ref #635807
2018-11-26 19:15:48 [DEBUG   ] summary = ERROR
2018-11-26 19:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635807  13 58448 69348777 A  ERRORERROR'
2018-11-26 19:15:48 [DEBUG   ] Got command RPT from MCS: ref #635808
2018-11-26 19:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635808  77 58448 69348778 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:16:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:16:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:16:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:16:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:16:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:16:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:16:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:16:22 [DEBUG   ] Finished updating board flags in 1.643 seconds
2018-11-26 19:16:25 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:16:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:17:05 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 19:17:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:17:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:17:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:17:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:17:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:17:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:17:22 [DEBUG   ] Finished updating board flags in 1.684 seconds
2018-11-26 19:17:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:17:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:17:47 [DEBUG   ] Got command RPT from MCS: ref #635828
2018-11-26 19:17:47 [DEBUG   ] summary = ERROR
2018-11-26 19:17:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635828  13 58448 69467377 A  ERRORERROR'
2018-11-26 19:17:47 [DEBUG   ] Got command RPT from MCS: ref #635829
2018-11-26 19:17:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:17:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635829  77 58448 69467581 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:17:47 [DEBUG   ] Got command RPT from MCS: ref #635830
2018-11-26 19:17:47 [DEBUG   ] summary = ERROR
2018-11-26 19:17:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635830  13 58448 69467785 A  ERRORERROR'
2018-11-26 19:17:47 [DEBUG   ] Got command RPT from MCS: ref #635831
2018-11-26 19:17:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:17:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635831  77 58448 69467977 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:17:48 [DEBUG   ] Got command RPT from MCS: ref #635832
2018-11-26 19:17:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635832  12 58448 69468181 A  ERRORB–™š'
2018-11-26 19:18:05 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 19:18:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:18:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:18:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:18:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:18:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:18:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:18:22 [DEBUG   ] Finished updating board flags in 1.475 seconds
2018-11-26 19:18:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 19:18:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:19:05 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:19:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:19:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:19:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:19:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:19:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:19:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:19:22 [DEBUG   ] Finished updating board flags in 1.500 seconds
2018-11-26 19:19:25 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 19:19:45 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2018-11-26 19:20:05 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 19:20:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:20:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:20:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:20:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:20:22 [WARNING ] (13, 1, 'bot')
2018-11-26 19:20:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:20:22 [DEBUG   ] Finished updating board flags in 1.498 seconds
2018-11-26 19:20:25 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 19:20:46 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 19:20:48 [DEBUG   ] Got command RPT from MCS: ref #635837
2018-11-26 19:20:48 [DEBUG   ] summary = ERROR
2018-11-26 19:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635837  13 58448 69648781 A  ERRORERROR'
2018-11-26 19:20:48 [DEBUG   ] Got command RPT from MCS: ref #635838
2018-11-26 19:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635838  77 58448 69648782 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:21:06 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 19:21:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:21:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:21:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:21:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:21:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:21:23 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:21:23 [DEBUG   ] Finished updating board flags in 2.117 seconds
2018-11-26 19:21:26 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 19:21:46 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 19:22:06 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:22:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:22:23 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:22:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:22:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:22:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:22:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:22:23 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:22:23 [DEBUG   ] Finished updating board flags in 2.011 seconds
2018-11-26 19:22:26 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 19:22:46 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 19:22:47 [DEBUG   ] Got command RPT from MCS: ref #635860
2018-11-26 19:22:47 [DEBUG   ] summary = ERROR
2018-11-26 19:22:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635860  13 58448 69767461 A  ERRORERROR'
2018-11-26 19:22:47 [DEBUG   ] Got command RPT from MCS: ref #635861
2018-11-26 19:22:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:22:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635861  77 58448 69767665 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:22:47 [DEBUG   ] Got command RPT from MCS: ref #635862
2018-11-26 19:22:47 [DEBUG   ] summary = ERROR
2018-11-26 19:22:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635862  13 58448 69767869 A  ERRORERROR'
2018-11-26 19:22:48 [DEBUG   ] Got command RPT from MCS: ref #635864
2018-11-26 19:22:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635864  77 58448 69768061 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:22:48 [DEBUG   ] Got command RPT from MCS: ref #635865
2018-11-26 19:22:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635865  12 58448 69768265 A  ERRORB•™š'
2018-11-26 19:23:06 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 19:23:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:23:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:23:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:23:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:23:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:23:23 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:23:23 [DEBUG   ] Finished updating board flags in 1.975 seconds
2018-11-26 19:23:26 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 19:23:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 19:24:06 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 19:24:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:24:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:24:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:24:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:24:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:24:23 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:24:23 [DEBUG   ] Finished updating board flags in 1.825 seconds
2018-11-26 19:24:26 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 19:24:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 19:25:06 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:25:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:25:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:25:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:25:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:25:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:25:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:25:23 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:25:23 [DEBUG   ] Finished updating board flags in 1.779 seconds
2018-11-26 19:25:26 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 19:25:46 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:25:48 [DEBUG   ] Got command RPT from MCS: ref #635874
2018-11-26 19:25:48 [DEBUG   ] summary = ERROR
2018-11-26 19:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635874  13 58448 69948785 A  ERRORERROR'
2018-11-26 19:25:48 [DEBUG   ] Got command RPT from MCS: ref #635875
2018-11-26 19:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635875  77 58448 69948786 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:26:06 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 19:26:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:26:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:26:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:26:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:26:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:26:23 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:26:23 [DEBUG   ] Finished updating board flags in 1.717 seconds
2018-11-26 19:26:26 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:26:46 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 19:27:06 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 19:27:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:27:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:27:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:27:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:27:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:27:23 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:27:23 [DEBUG   ] Finished updating board flags in 1.482 seconds
2018-11-26 19:27:27 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 19:27:46 [DEBUG   ] Finished updating ant stats in 0.406 seconds
2018-11-26 19:27:47 [DEBUG   ] Got command RPT from MCS: ref #635891
2018-11-26 19:27:47 [DEBUG   ] summary = ERROR
2018-11-26 19:27:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635891  13 58448 70067549 A  ERRORERROR'
2018-11-26 19:27:47 [DEBUG   ] Got command RPT from MCS: ref #635892
2018-11-26 19:27:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:27:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635892  77 58448 70067753 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:27:47 [DEBUG   ] Got command RPT from MCS: ref #635893
2018-11-26 19:27:47 [DEBUG   ] summary = ERROR
2018-11-26 19:27:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635893  13 58448 70067945 A  ERRORERROR'
2018-11-26 19:27:48 [DEBUG   ] Got command RPT from MCS: ref #635894
2018-11-26 19:27:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635894  77 58448 70068149 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:27:48 [DEBUG   ] Got command RPT from MCS: ref #635895
2018-11-26 19:27:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635895  12 58448 70068353 A  ERRORB•™š'
2018-11-26 19:28:06 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 19:28:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:28:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:28:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:28:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:28:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:28:23 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:28:23 [DEBUG   ] Finished updating board flags in 1.420 seconds
2018-11-26 19:28:26 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 19:28:47 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 19:29:07 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 19:29:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:29:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:29:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:29:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:29:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:29:23 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:29:23 [DEBUG   ] Finished updating board flags in 1.386 seconds
2018-11-26 19:29:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 19:29:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 19:30:07 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 19:30:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:30:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:30:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:30:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:30:23 [WARNING ] (13, 1, 'bot')
2018-11-26 19:30:23 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:30:23 [DEBUG   ] Finished updating board flags in 1.413 seconds
2018-11-26 19:30:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 19:30:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 19:30:48 [DEBUG   ] Got command RPT from MCS: ref #635900
2018-11-26 19:30:48 [DEBUG   ] summary = ERROR
2018-11-26 19:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635900  13 58448 70248781 A  ERRORERROR'
2018-11-26 19:30:48 [DEBUG   ] Got command RPT from MCS: ref #635901
2018-11-26 19:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635901  77 58448 70248782 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:31:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 19:31:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:31:24 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:31:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:31:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:31:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:31:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:31:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:31:24 [DEBUG   ] Finished updating board flags in 2.205 seconds
2018-11-26 19:31:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 19:31:47 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 19:32:07 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 19:32:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:32:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:32:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:32:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:32:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:32:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:32:24 [DEBUG   ] Finished updating board flags in 2.305 seconds
2018-11-26 19:32:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 19:32:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 19:32:47 [DEBUG   ] Got command RPT from MCS: ref #635923
2018-11-26 19:32:47 [DEBUG   ] summary = ERROR
2018-11-26 19:32:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635923  13 58448 70367641 A  ERRORERROR'
2018-11-26 19:32:47 [DEBUG   ] Got command RPT from MCS: ref #635924
2018-11-26 19:32:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:32:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635924  77 58448 70367833 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:32:48 [DEBUG   ] Got command RPT from MCS: ref #635925
2018-11-26 19:32:48 [DEBUG   ] summary = ERROR
2018-11-26 19:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635925  13 58448 70368037 A  ERRORERROR'
2018-11-26 19:32:48 [DEBUG   ] Got command RPT from MCS: ref #635927
2018-11-26 19:32:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635927  77 58448 70368241 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:32:48 [DEBUG   ] Got command RPT from MCS: ref #635929
2018-11-26 19:32:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635929  12 58448 70368445 A  ERRORB”™š'
2018-11-26 19:33:07 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 19:33:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:33:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:33:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:33:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:33:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:33:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:33:24 [DEBUG   ] Finished updating board flags in 2.081 seconds
2018-11-26 19:33:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 19:33:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 19:34:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 19:34:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:34:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:34:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:34:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:34:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:34:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:34:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:34:24 [DEBUG   ] Finished updating board flags in 2.023 seconds
2018-11-26 19:34:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 19:34:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 19:35:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 19:35:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:35:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:35:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:35:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:35:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:35:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:35:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:35:24 [DEBUG   ] Finished updating board flags in 1.979 seconds
2018-11-26 19:35:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 19:35:47 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 19:35:48 [DEBUG   ] Got command RPT from MCS: ref #635937
2018-11-26 19:35:48 [DEBUG   ] summary = ERROR
2018-11-26 19:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635937  13 58448 70548789 A  ERRORERROR'
2018-11-26 19:35:48 [DEBUG   ] Got command RPT from MCS: ref #635938
2018-11-26 19:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635938  77 58448 70548790 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:36:07 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 19:36:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:36:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:36:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:36:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:36:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:36:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:36:24 [DEBUG   ] Finished updating board flags in 1.831 seconds
2018-11-26 19:36:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:36:47 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:37:07 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:37:23 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 19:37:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:37:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:37:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:37:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:37:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:37:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:37:24 [DEBUG   ] Finished updating board flags in 1.777 seconds
2018-11-26 19:37:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:37:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:37:47 [DEBUG   ] Got command RPT from MCS: ref #635958
2018-11-26 19:37:47 [DEBUG   ] summary = ERROR
2018-11-26 19:37:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635958  13 58448 70667725 A  ERRORERROR'
2018-11-26 19:37:47 [DEBUG   ] Got command RPT from MCS: ref #635959
2018-11-26 19:37:47 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:37:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635959  77 58448 70667929 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:37:48 [DEBUG   ] Got command RPT from MCS: ref #635960
2018-11-26 19:37:48 [DEBUG   ] summary = ERROR
2018-11-26 19:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635960  13 58448 70668121 A  ERRORERROR'
2018-11-26 19:37:48 [DEBUG   ] Got command RPT from MCS: ref #635961
2018-11-26 19:37:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635961  77 58448 70668325 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:37:48 [DEBUG   ] Got command RPT from MCS: ref #635962
2018-11-26 19:37:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635962  12 58448 70668529 A  ERRORB•™š'
2018-11-26 19:38:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:38:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:38:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:38:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:38:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:38:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:38:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:38:24 [DEBUG   ] Finished updating board flags in 1.715 seconds
2018-11-26 19:38:27 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:38:47 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:39:07 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:39:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:39:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:39:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:39:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:39:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:39:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:39:24 [DEBUG   ] Finished updating board flags in 1.492 seconds
2018-11-26 19:39:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:39:47 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:40:07 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:40:23 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:40:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:40:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:40:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:40:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:40:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:40:24 [DEBUG   ] Finished updating board flags in 1.501 seconds
2018-11-26 19:40:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:40:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:40:48 [DEBUG   ] Got command RPT from MCS: ref #635967
2018-11-26 19:40:48 [DEBUG   ] summary = ERROR
2018-11-26 19:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635967  13 58448 70848789 A  ERRORERROR'
2018-11-26 19:40:48 [DEBUG   ] Got command RPT from MCS: ref #635968
2018-11-26 19:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635968  77 58448 70848790 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:41:07 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:41:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:41:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:41:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:41:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:41:24 [WARNING ] (13, 1, 'bot')
2018-11-26 19:41:24 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:41:24 [DEBUG   ] Finished updating board flags in 1.515 seconds
2018-11-26 19:41:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:41:47 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:42:07 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:42:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 19:42:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:42:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:42:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:42:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:42:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:42:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:42:25 [DEBUG   ] Finished updating board flags in 2.234 seconds
2018-11-26 19:42:27 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:42:47 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:42:47 [DEBUG   ] Got command RPT from MCS: ref #635986
2018-11-26 19:42:47 [DEBUG   ] summary = ERROR
2018-11-26 19:42:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635986  13 58448 70967813 A  ERRORERROR'
2018-11-26 19:42:48 [DEBUG   ] Got command RPT from MCS: ref #635987
2018-11-26 19:42:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635987  77 58448 70968017 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:42:48 [DEBUG   ] Got command RPT from MCS: ref #635988
2018-11-26 19:42:48 [DEBUG   ] summary = ERROR
2018-11-26 19:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635988  13 58448 70968209 A  ERRORERROR'
2018-11-26 19:42:48 [DEBUG   ] Got command RPT from MCS: ref #635990
2018-11-26 19:42:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635990  77 58448 70968413 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:42:48 [DEBUG   ] Got command RPT from MCS: ref #635992
2018-11-26 19:42:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   635992  12 58448 70968617 A  ERRORB—™š'
2018-11-26 19:43:07 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:43:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:43:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 19:43:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:43:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:43:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:43:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:43:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:43:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:43:25 [DEBUG   ] Finished updating board flags in 2.291 seconds
2018-11-26 19:43:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:43:47 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 19:44:07 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:44:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:44:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:44:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:44:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:44:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:44:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:44:25 [DEBUG   ] Finished updating board flags in 2.070 seconds
2018-11-26 19:44:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:44:47 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:45:07 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:45:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:45:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:45:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:45:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:45:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:45:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:45:25 [DEBUG   ] Finished updating board flags in 2.024 seconds
2018-11-26 19:45:27 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:45:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:45:48 [DEBUG   ] Got command RPT from MCS: ref #636000
2018-11-26 19:45:48 [DEBUG   ] summary = ERROR
2018-11-26 19:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636000  13 58448 71148789 A  ERRORERROR'
2018-11-26 19:45:48 [DEBUG   ] Got command RPT from MCS: ref #636001
2018-11-26 19:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636001  77 58448 71148790 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:46:07 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:46:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:46:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:46:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:46:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:46:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:46:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:46:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:46:25 [DEBUG   ] Finished updating board flags in 2.010 seconds
2018-11-26 19:46:27 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 19:46:47 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 19:47:07 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 19:47:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:47:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:47:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:47:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:47:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:47:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:47:25 [DEBUG   ] Finished updating board flags in 1.977 seconds
2018-11-26 19:47:27 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2018-11-26 19:47:47 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:47:47 [DEBUG   ] Got command RPT from MCS: ref #636021
2018-11-26 19:47:47 [DEBUG   ] summary = ERROR
2018-11-26 19:47:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636021  13 58448 71267901 A  ERRORERROR'
2018-11-26 19:47:48 [DEBUG   ] Got command RPT from MCS: ref #636022
2018-11-26 19:47:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636022  77 58448 71268093 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:47:48 [DEBUG   ] Got command RPT from MCS: ref #636023
2018-11-26 19:47:48 [DEBUG   ] summary = ERROR
2018-11-26 19:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636023  13 58448 71268297 A  ERRORERROR'
2018-11-26 19:47:48 [DEBUG   ] Got command RPT from MCS: ref #636024
2018-11-26 19:47:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636024  77 58448 71268501 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:47:48 [DEBUG   ] Got command RPT from MCS: ref #636025
2018-11-26 19:47:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636025  12 58448 71268693 A  ERRORB–™š'
2018-11-26 19:48:07 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:48:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:48:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:48:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:48:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:48:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:48:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:48:25 [DEBUG   ] Finished updating board flags in 1.806 seconds
2018-11-26 19:48:27 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:48:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:49:07 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 19:49:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:49:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:49:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:49:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:49:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:49:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:49:25 [DEBUG   ] Finished updating board flags in 1.821 seconds
2018-11-26 19:49:27 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:49:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:50:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:50:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:50:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:50:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:50:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:50:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:50:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:50:25 [DEBUG   ] Finished updating board flags in 1.880 seconds
2018-11-26 19:50:27 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:50:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 19:50:48 [DEBUG   ] Got command RPT from MCS: ref #636030
2018-11-26 19:50:48 [DEBUG   ] summary = ERROR
2018-11-26 19:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636030  13 58448 71448785 A  ERRORERROR'
2018-11-26 19:50:48 [DEBUG   ] Got command RPT from MCS: ref #636031
2018-11-26 19:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636031  77 58448 71448786 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:51:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 19:51:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:51:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:51:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:51:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:51:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:51:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:51:25 [DEBUG   ] Finished updating board flags in 1.716 seconds
2018-11-26 19:51:27 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 19:51:47 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 19:52:07 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 19:52:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:52:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:52:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:52:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:52:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:52:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:52:25 [DEBUG   ] Finished updating board flags in 1.592 seconds
2018-11-26 19:52:28 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 19:52:47 [DEBUG   ] Got command RPT from MCS: ref #636053
2018-11-26 19:52:47 [DEBUG   ] summary = ERROR
2018-11-26 19:52:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636053  13 58448 71567985 A  ERRORERROR'
2018-11-26 19:52:48 [DEBUG   ] Got command RPT from MCS: ref #636054
2018-11-26 19:52:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636054  77 58448 71568177 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:52:48 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 19:52:48 [DEBUG   ] Got command RPT from MCS: ref #636055
2018-11-26 19:52:48 [DEBUG   ] summary = ERROR
2018-11-26 19:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636055  13 58448 71568381 A  ERRORERROR'
2018-11-26 19:52:48 [DEBUG   ] Got command RPT from MCS: ref #636057
2018-11-26 19:52:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636057  77 58448 71568585 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:52:48 [DEBUG   ] Got command RPT from MCS: ref #636058
2018-11-26 19:52:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:52:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636058  12 58448 71568789 A  ERRORB•™š'
2018-11-26 19:53:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 19:53:24 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:53:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:53:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:53:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:53:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:53:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:53:25 [DEBUG   ] Finished updating board flags in 1.394 seconds
2018-11-26 19:53:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 19:53:48 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 19:54:08 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 19:54:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:54:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:54:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:54:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:54:25 [WARNING ] (13, 1, 'bot')
2018-11-26 19:54:25 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:54:25 [DEBUG   ] Finished updating board flags in 1.371 seconds
2018-11-26 19:54:28 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 19:54:48 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:55:08 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:55:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:55:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 19:55:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:55:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:55:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:55:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:55:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:55:26 [DEBUG   ] Finished updating board flags in 2.039 seconds
2018-11-26 19:55:28 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 19:55:48 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:55:48 [DEBUG   ] Got command RPT from MCS: ref #636067
2018-11-26 19:55:48 [DEBUG   ] summary = ERROR
2018-11-26 19:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636067  13 58448 71748789 A  ERRORERROR'
2018-11-26 19:55:48 [DEBUG   ] Got command RPT from MCS: ref #636068
2018-11-26 19:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636068  77 58448 71748790 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:56:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:56:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:56:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:56:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:56:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:56:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:56:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:56:26 [DEBUG   ] Finished updating board flags in 2.009 seconds
2018-11-26 19:56:28 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:56:48 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 19:57:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:57:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:57:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:57:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:57:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:57:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:57:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:57:26 [DEBUG   ] Finished updating board flags in 2.032 seconds
2018-11-26 19:57:28 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:57:48 [DEBUG   ] Got command RPT from MCS: ref #636084
2018-11-26 19:57:48 [DEBUG   ] summary = ERROR
2018-11-26 19:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636084  13 58448 71868065 A  ERRORERROR'
2018-11-26 19:57:48 [DEBUG   ] Got command RPT from MCS: ref #636085
2018-11-26 19:57:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636085  77 58448 71868269 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:57:48 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:57:48 [DEBUG   ] Got command RPT from MCS: ref #636086
2018-11-26 19:57:48 [DEBUG   ] summary = ERROR
2018-11-26 19:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636086  13 58448 71868461 A  ERRORERROR'
2018-11-26 19:57:48 [DEBUG   ] Got command RPT from MCS: ref #636087
2018-11-26 19:57:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 19:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636087  77 58448 71868665 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 19:57:48 [DEBUG   ] Got command RPT from MCS: ref #636088
2018-11-26 19:57:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 19:57:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636088  12 58448 71868869 A  ERRORB”™š'
2018-11-26 19:58:08 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 19:58:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:58:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:58:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:58:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:58:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:58:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:58:26 [DEBUG   ] Finished updating board flags in 2.022 seconds
2018-11-26 19:58:28 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:58:48 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 19:59:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 19:59:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 19:59:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:59:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:59:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:59:26 [WARNING ] (13, 1, 'bot')
2018-11-26 19:59:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 19:59:26 [DEBUG   ] Finished updating board flags in 1.967 seconds
2018-11-26 19:59:28 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 19:59:48 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:00:08 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 20:00:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:00:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:00:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:00:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:00:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:00:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:00:26 [DEBUG   ] Finished updating board flags in 1.843 seconds
2018-11-26 20:00:28 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:00:48 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 20:00:48 [DEBUG   ] Got command RPT from MCS: ref #636093
2018-11-26 20:00:48 [DEBUG   ] summary = ERROR
2018-11-26 20:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636093  13 58448 72048793 A  ERRORERROR'
2018-11-26 20:00:48 [DEBUG   ] Got command RPT from MCS: ref #636094
2018-11-26 20:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636094  77 58448 72048794 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:01:08 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:01:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:01:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:01:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:01:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:01:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:01:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:01:26 [DEBUG   ] Finished updating board flags in 1.761 seconds
2018-11-26 20:01:28 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 20:01:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:02:08 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 20:02:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:02:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:02:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:02:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:02:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:02:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:02:26 [DEBUG   ] Finished updating board flags in 1.624 seconds
2018-11-26 20:02:28 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 20:02:48 [DEBUG   ] Got command RPT from MCS: ref #636116
2018-11-26 20:02:48 [DEBUG   ] summary = ERROR
2018-11-26 20:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636116  13 58448 72168157 A  ERRORERROR'
2018-11-26 20:02:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 20:02:48 [DEBUG   ] Got command RPT from MCS: ref #636117
2018-11-26 20:02:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636117  77 58448 72168349 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:02:48 [DEBUG   ] Got command RPT from MCS: ref #636118
2018-11-26 20:02:48 [DEBUG   ] summary = ERROR
2018-11-26 20:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636118  13 58448 72168553 A  ERRORERROR'
2018-11-26 20:02:48 [DEBUG   ] Got command RPT from MCS: ref #636120
2018-11-26 20:02:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636120  77 58448 72168757 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:02:48 [DEBUG   ] Got command RPT from MCS: ref #636121
2018-11-26 20:02:48 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:02:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636121  12 58448 72168961 A  ERRORB—™š'
2018-11-26 20:03:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 20:03:25 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 20:03:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:03:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:03:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:03:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:03:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:03:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:03:26 [DEBUG   ] Finished updating board flags in 1.543 seconds
2018-11-26 20:03:28 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 20:03:48 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 20:04:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:04:25 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 20:04:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:04:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 20:04:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:04:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:04:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:04:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:04:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:04:26 [DEBUG   ] Finished updating board flags in 1.439 seconds
2018-11-26 20:04:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:04:48 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 20:05:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 20:05:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 20:05:25 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:05:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 20:05:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:05:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:05:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:05:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:05:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:05:26 [DEBUG   ] Finished updating board flags in 1.489 seconds
2018-11-26 20:05:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 20:05:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:05:48 [DEBUG   ] Got command RPT from MCS: ref #636130
2018-11-26 20:05:48 [DEBUG   ] summary = ERROR
2018-11-26 20:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636130  13 58448 72348793 A  ERRORERROR'
2018-11-26 20:05:48 [DEBUG   ] Got command RPT from MCS: ref #636131
2018-11-26 20:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636131  77 58448 72348794 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:06:08 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 20:06:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 20:06:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:06:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:06:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:06:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:06:26 [WARNING ] (13, 1, 'bot')
2018-11-26 20:06:26 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:06:26 [DEBUG   ] Finished updating board flags in 1.519 seconds
2018-11-26 20:06:28 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 20:06:48 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 20:07:09 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 20:07:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:07:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:07:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:07:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:07:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:07:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:07:27 [DEBUG   ] Finished updating board flags in 2.233 seconds
2018-11-26 20:07:29 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 20:07:48 [DEBUG   ] Got command RPT from MCS: ref #636151
2018-11-26 20:07:48 [DEBUG   ] summary = ERROR
2018-11-26 20:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636151  13 58448 72468233 A  ERRORERROR'
2018-11-26 20:07:48 [DEBUG   ] Got command RPT from MCS: ref #636152
2018-11-26 20:07:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636152  77 58448 72468437 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:07:48 [DEBUG   ] Got command RPT from MCS: ref #636153
2018-11-26 20:07:48 [DEBUG   ] summary = ERROR
2018-11-26 20:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636153  13 58448 72468641 A  ERRORERROR'
2018-11-26 20:07:48 [DEBUG   ] Got command RPT from MCS: ref #636154
2018-11-26 20:07:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:07:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636154  77 58448 72468833 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:07:49 [DEBUG   ] Got command RPT from MCS: ref #636155
2018-11-26 20:07:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636155  12 58448 72469037 A  ERRORB—™š'
2018-11-26 20:07:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 20:08:09 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:08:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:08:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:08:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:08:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:08:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:08:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:08:27 [DEBUG   ] Finished updating board flags in 2.128 seconds
2018-11-26 20:08:29 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 20:08:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 20:09:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 20:09:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:09:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:09:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:09:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:09:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:09:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:09:27 [DEBUG   ] Finished updating board flags in 2.044 seconds
2018-11-26 20:09:29 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:09:49 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 20:10:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 20:10:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:10:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:10:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:10:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:10:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:10:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:10:27 [DEBUG   ] Finished updating board flags in 2.057 seconds
2018-11-26 20:10:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 20:10:48 [DEBUG   ] Got command RPT from MCS: ref #636160
2018-11-26 20:10:48 [DEBUG   ] summary = ERROR
2018-11-26 20:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636160  13 58448 72648797 A  ERRORERROR'
2018-11-26 20:10:48 [DEBUG   ] Got command RPT from MCS: ref #636161
2018-11-26 20:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636161  77 58448 72648798 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:10:49 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:11:09 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 20:11:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:11:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:11:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:11:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:11:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:11:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:11:27 [DEBUG   ] Finished updating board flags in 1.906 seconds
2018-11-26 20:11:29 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 20:11:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 20:12:09 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:12:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:12:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:12:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:12:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:12:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:12:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:12:27 [DEBUG   ] Finished updating board flags in 1.874 seconds
2018-11-26 20:12:29 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 20:12:48 [DEBUG   ] Got command RPT from MCS: ref #636179
2018-11-26 20:12:48 [DEBUG   ] summary = ERROR
2018-11-26 20:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636179  13 58448 72768318 A  ERRORERROR'
2018-11-26 20:12:48 [DEBUG   ] Got command RPT from MCS: ref #636180
2018-11-26 20:12:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636180  77 58448 72768533 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:12:48 [DEBUG   ] Got command RPT from MCS: ref #636181
2018-11-26 20:12:48 [DEBUG   ] summary = ERROR
2018-11-26 20:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636181  13 58448 72768725 A  ERRORERROR'
2018-11-26 20:12:48 [DEBUG   ] Got command RPT from MCS: ref #636183
2018-11-26 20:12:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636183  77 58448 72768929 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:12:49 [DEBUG   ] Got command RPT from MCS: ref #636184
2018-11-26 20:12:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636184  12 58448 72769121 A  ERRORB–™š'
2018-11-26 20:12:49 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 20:13:09 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 20:13:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:13:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:13:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:13:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:13:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:13:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:13:27 [DEBUG   ] Finished updating board flags in 1.740 seconds
2018-11-26 20:13:29 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:13:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:14:09 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 20:14:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:14:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:14:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:14:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:14:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:14:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:14:27 [DEBUG   ] Finished updating board flags in 1.603 seconds
2018-11-26 20:14:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 20:14:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:15:09 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 20:15:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:15:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:15:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:15:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:15:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:15:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:15:27 [DEBUG   ] Finished updating board flags in 1.657 seconds
2018-11-26 20:15:29 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2018-11-26 20:15:48 [DEBUG   ] Got command RPT from MCS: ref #636193
2018-11-26 20:15:48 [DEBUG   ] summary = ERROR
2018-11-26 20:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636193  13 58448 72948797 A  ERRORERROR'
2018-11-26 20:15:48 [DEBUG   ] Got command RPT from MCS: ref #636194
2018-11-26 20:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636194  77 58448 72948798 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:15:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:16:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:16:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:16:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:16:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:16:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:16:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:16:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:16:27 [DEBUG   ] Finished updating board flags in 1.545 seconds
2018-11-26 20:16:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:16:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:17:09 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:17:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:17:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:17:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:17:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:17:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:17:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:17:27 [DEBUG   ] Finished updating board flags in 1.418 seconds
2018-11-26 20:17:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:17:48 [DEBUG   ] Got command RPT from MCS: ref #636214
2018-11-26 20:17:48 [DEBUG   ] summary = ERROR
2018-11-26 20:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636214  13 58448 73068413 A  ERRORERROR'
2018-11-26 20:17:48 [DEBUG   ] Got command RPT from MCS: ref #636215
2018-11-26 20:17:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636215  77 58448 73068605 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:17:48 [DEBUG   ] Got command RPT from MCS: ref #636216
2018-11-26 20:17:48 [DEBUG   ] summary = ERROR
2018-11-26 20:17:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636216  13 58448 73068813 A  ERRORERROR'
2018-11-26 20:17:49 [DEBUG   ] Got command RPT from MCS: ref #636217
2018-11-26 20:17:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636217  77 58448 73069005 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:17:49 [DEBUG   ] Got command RPT from MCS: ref #636218
2018-11-26 20:17:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636218  12 58448 73069209 A  ERRORB•™š'
2018-11-26 20:17:49 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 20:18:09 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:18:26 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:18:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:18:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:18:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:18:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:18:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:18:27 [DEBUG   ] Finished updating board flags in 1.495 seconds
2018-11-26 20:18:29 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:18:49 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:19:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:19:27 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:19:27 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 20:19:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:19:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:19:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:19:27 [WARNING ] (13, 1, 'bot')
2018-11-26 20:19:27 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:19:27 [DEBUG   ] Finished updating board flags in 1.547 seconds
2018-11-26 20:19:29 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:19:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:20:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:20:27 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:20:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 20:20:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:20:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:20:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:20:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:20:28 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:20:28 [DEBUG   ] Finished updating board flags in 2.250 seconds
2018-11-26 20:20:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:20:48 [DEBUG   ] Got command RPT from MCS: ref #636223
2018-11-26 20:20:48 [DEBUG   ] summary = ERROR
2018-11-26 20:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636223  13 58448 73248789 A  ERRORERROR'
2018-11-26 20:20:48 [DEBUG   ] Got command RPT from MCS: ref #636224
2018-11-26 20:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636224  77 58448 73248790 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:20:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 20:21:09 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:21:27 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:21:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:21:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:21:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:21:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:21:28 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:21:28 [DEBUG   ] Finished updating board flags in 2.271 seconds
2018-11-26 20:21:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:21:49 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 20:22:09 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 20:22:27 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:22:29 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 20:22:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:22:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:22:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:22:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:22:29 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:22:29 [DEBUG   ] Finished updating board flags in 2.751 seconds
2018-11-26 20:22:48 [DEBUG   ] Got command RPT from MCS: ref #636246
2018-11-26 20:22:48 [DEBUG   ] summary = ERROR
2018-11-26 20:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636246  13 58448 73368493 A  ERRORERROR'
2018-11-26 20:22:48 [DEBUG   ] Got command RPT from MCS: ref #636247
2018-11-26 20:22:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636247  77 58448 73368693 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:22:48 [DEBUG   ] Got command RPT from MCS: ref #636248
2018-11-26 20:22:48 [DEBUG   ] summary = ERROR
2018-11-26 20:22:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636248  13 58448 73368897 A  ERRORERROR'
2018-11-26 20:22:49 [DEBUG   ] Got command RPT from MCS: ref #636250
2018-11-26 20:22:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636250  77 58448 73369101 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:22:49 [DEBUG   ] Got command RPT from MCS: ref #636252
2018-11-26 20:22:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636252  12 58448 73369293 A  ERRORB•™š'
2018-11-26 20:22:49 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:23:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:23:27 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:23:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:23:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:23:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:23:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:23:28 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:23:28 [DEBUG   ] Finished updating board flags in 1.926 seconds
2018-11-26 20:23:29 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 20:23:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:24:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:24:27 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:24:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 20:24:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:24:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:24:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:24:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:24:28 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:24:28 [DEBUG   ] Finished updating board flags in 1.756 seconds
2018-11-26 20:24:29 [DEBUG   ] Finished updating ant stats in 0.623 seconds
2018-11-26 20:24:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:25:09 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:25:27 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:25:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:25:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:25:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:25:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:25:28 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:25:28 [DEBUG   ] Finished updating board flags in 1.630 seconds
2018-11-26 20:25:29 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 20:25:48 [DEBUG   ] Got command RPT from MCS: ref #636260
2018-11-26 20:25:48 [DEBUG   ] summary = ERROR
2018-11-26 20:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636260  13 58448 73548793 A  ERRORERROR'
2018-11-26 20:25:48 [DEBUG   ] Got command RPT from MCS: ref #636261
2018-11-26 20:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636261  77 58448 73548794 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:25:49 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 20:26:09 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:26:27 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:26:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:26:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:26:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:26:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:26:28 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:26:28 [DEBUG   ] Finished updating board flags in 1.596 seconds
2018-11-26 20:26:29 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:26:49 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 20:27:09 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 20:27:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:27:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:27:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:27:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:27:28 [WARNING ] (13, 1, 'bot')
2018-11-26 20:27:28 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:27:28 [DEBUG   ] Finished updating board flags in 1.599 seconds
2018-11-26 20:27:29 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:27:48 [DEBUG   ] Got command RPT from MCS: ref #636277
2018-11-26 20:27:48 [DEBUG   ] summary = ERROR
2018-11-26 20:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636277  13 58448 73668577 A  ERRORERROR'
2018-11-26 20:27:48 [DEBUG   ] Got command RPT from MCS: ref #636278
2018-11-26 20:27:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636278  77 58448 73668781 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:27:48 [DEBUG   ] Got command RPT from MCS: ref #636279
2018-11-26 20:27:48 [DEBUG   ] summary = ERROR
2018-11-26 20:27:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636279  13 58448 73668989 A  ERRORERROR'
2018-11-26 20:27:49 [DEBUG   ] Got command RPT from MCS: ref #636280
2018-11-26 20:27:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636280  77 58448 73669185 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:27:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:27:49 [DEBUG   ] Got command RPT from MCS: ref #636281
2018-11-26 20:27:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636281  12 58448 73669389 A  ERRORB•™š'
2018-11-26 20:28:09 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 20:28:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:28:29 [DEBUG   ] Finished updating ant stats in 0.647 seconds
2018-11-26 20:28:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 20:28:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:28:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:28:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:28:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:28:29 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:28:29 [DEBUG   ] Finished updating board flags in 2.244 seconds
2018-11-26 20:28:49 [DEBUG   ] Finished updating ant stats in 0.558 seconds
2018-11-26 20:29:09 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 20:29:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:29:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:29:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:29:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:29:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:29:29 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:29:29 [DEBUG   ] Finished updating board flags in 2.217 seconds
2018-11-26 20:29:30 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-26 20:29:50 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 20:30:10 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:30:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:30:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-26 20:30:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:30:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:30:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:30:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:30:29 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:30:29 [DEBUG   ] Finished updating board flags in 2.170 seconds
2018-11-26 20:30:30 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 20:30:48 [DEBUG   ] Got command RPT from MCS: ref #636286
2018-11-26 20:30:48 [DEBUG   ] summary = ERROR
2018-11-26 20:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636286  13 58448 73848801 A  ERRORERROR'
2018-11-26 20:30:48 [DEBUG   ] Got command RPT from MCS: ref #636287
2018-11-26 20:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636287  77 58448 73848802 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:30:50 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 20:31:10 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 20:31:28 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 20:31:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:31:30 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 20:31:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:31:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:31:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:31:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:31:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:31:30 [DEBUG   ] Finished updating board flags in 3.186 seconds
2018-11-26 20:31:50 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 20:32:10 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 20:32:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:32:29 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 20:32:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:32:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:32:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:32:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:32:29 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:32:29 [DEBUG   ] Finished updating board flags in 1.907 seconds
2018-11-26 20:32:30 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 20:32:48 [DEBUG   ] Got command RPT from MCS: ref #636309
2018-11-26 20:32:48 [DEBUG   ] summary = ERROR
2018-11-26 20:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636309  13 58448 73968665 A  ERRORERROR'
2018-11-26 20:32:48 [DEBUG   ] Got command RPT from MCS: ref #636310
2018-11-26 20:32:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:32:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636310  77 58448 73968869 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:32:49 [DEBUG   ] Got command RPT from MCS: ref #636311
2018-11-26 20:32:49 [DEBUG   ] summary = ERROR
2018-11-26 20:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636311  13 58448 73969073 A  ERRORERROR'
2018-11-26 20:32:49 [DEBUG   ] Got command RPT from MCS: ref #636313
2018-11-26 20:32:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636313  77 58448 73969265 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:32:49 [DEBUG   ] Got command RPT from MCS: ref #636315
2018-11-26 20:32:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636315  12 58448 73969469 A  ERRORB—™š'
2018-11-26 20:32:50 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:33:10 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 20:33:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:33:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 20:33:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:33:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:33:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:33:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:33:29 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:33:29 [DEBUG   ] Finished updating board flags in 1.712 seconds
2018-11-26 20:33:30 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 20:33:50 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 20:34:10 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:34:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:34:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:34:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:34:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:34:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:34:29 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:34:29 [DEBUG   ] Finished updating board flags in 1.606 seconds
2018-11-26 20:34:30 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:34:50 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:35:10 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 20:35:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 20:35:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:35:30 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 20:35:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:35:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:35:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:35:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:35:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:35:30 [DEBUG   ] Finished updating board flags in 2.525 seconds
2018-11-26 20:35:48 [DEBUG   ] Got command RPT from MCS: ref #636328
2018-11-26 20:35:48 [DEBUG   ] summary = ERROR
2018-11-26 20:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636328  13 58448 74148793 A  ERRORERROR'
2018-11-26 20:35:48 [DEBUG   ] Got command RPT from MCS: ref #636329
2018-11-26 20:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636329  77 58448 74148794 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:35:50 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 20:36:10 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 20:36:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:36:30 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:36:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:36:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:36:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:36:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:36:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:36:30 [DEBUG   ] Finished updating board flags in 2.240 seconds
2018-11-26 20:36:50 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 20:37:10 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 20:37:28 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:37:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:37:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:37:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:37:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:37:29 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:37:29 [DEBUG   ] Finished updating board flags in 1.553 seconds
2018-11-26 20:37:30 [DEBUG   ] Finished updating ant stats in 0.795 seconds
2018-11-26 20:37:48 [DEBUG   ] Got command RPT from MCS: ref #636354
2018-11-26 20:37:48 [DEBUG   ] summary = ERROR
2018-11-26 20:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636354  13 58448 74268753 A  ERRORERROR'
2018-11-26 20:37:48 [DEBUG   ] Got command RPT from MCS: ref #636355
2018-11-26 20:37:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636355  77 58448 74268957 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:37:49 [DEBUG   ] Got command RPT from MCS: ref #636356
2018-11-26 20:37:49 [DEBUG   ] summary = ERROR
2018-11-26 20:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636356  13 58448 74269149 A  ERRORERROR'
2018-11-26 20:37:49 [DEBUG   ] Got command RPT from MCS: ref #636357
2018-11-26 20:37:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636357  77 58448 74269353 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:37:49 [DEBUG   ] Got command RPT from MCS: ref #636358
2018-11-26 20:37:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636358  12 58448 74269557 A  ERRORB–™š'
2018-11-26 20:37:50 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:38:10 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:38:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:38:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 20:38:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:38:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:38:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:38:29 [WARNING ] (13, 1, 'bot')
2018-11-26 20:38:29 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:38:29 [DEBUG   ] Finished updating board flags in 1.562 seconds
2018-11-26 20:38:30 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 20:38:50 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 20:39:11 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2018-11-26 20:39:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:39:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 20:39:30 [DEBUG   ] Finished updating ant stats in 0.647 seconds
2018-11-26 20:39:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 20:39:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:39:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:39:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:39:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:39:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:39:30 [DEBUG   ] Finished updating board flags in 2.179 seconds
2018-11-26 20:39:50 [DEBUG   ] Finished updating ant stats in 0.560 seconds
2018-11-26 20:40:11 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2018-11-26 20:40:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 20:40:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:40:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:40:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:40:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:40:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:40:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:40:30 [DEBUG   ] Finished updating board flags in 2.158 seconds
2018-11-26 20:40:31 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 20:40:48 [DEBUG   ] Got command RPT from MCS: ref #636363
2018-11-26 20:40:48 [DEBUG   ] summary = ERROR
2018-11-26 20:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636363  13 58448 74448801 A  ERRORERROR'
2018-11-26 20:40:48 [DEBUG   ] Got command RPT from MCS: ref #636364
2018-11-26 20:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636364  77 58448 74448802 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:40:51 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:41:11 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 20:41:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa10, chip 4
2018-11-26 20:41:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:41:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:41:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:41:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:41:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:41:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:41:30 [DEBUG   ] Finished updating board flags in 1.837 seconds
2018-11-26 20:41:31 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 20:41:51 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 20:42:11 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 20:42:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:42:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:42:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:42:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:42:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:42:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:42:30 [DEBUG   ] Finished updating board flags in 1.814 seconds
2018-11-26 20:42:31 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 20:42:48 [DEBUG   ] Got command RPT from MCS: ref #636382
2018-11-26 20:42:48 [DEBUG   ] summary = ERROR
2018-11-26 20:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636382  13 58448 74568841 A  ERRORERROR'
2018-11-26 20:42:49 [DEBUG   ] Got command RPT from MCS: ref #636383
2018-11-26 20:42:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636383  77 58448 74569033 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:42:49 [DEBUG   ] Got command RPT from MCS: ref #636384
2018-11-26 20:42:49 [DEBUG   ] summary = ERROR
2018-11-26 20:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636384  13 58448 74569237 A  ERRORERROR'
2018-11-26 20:42:49 [DEBUG   ] Got command RPT from MCS: ref #636386
2018-11-26 20:42:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636386  77 58448 74569441 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:42:49 [DEBUG   ] Got command RPT from MCS: ref #636387
2018-11-26 20:42:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636387  12 58448 74569645 A  ERRORB•™š'
2018-11-26 20:42:51 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 20:43:11 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:43:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:43:31 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 20:43:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:43:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:43:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:43:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:43:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:43:31 [DEBUG   ] Finished updating board flags in 2.919 seconds
2018-11-26 20:43:51 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 20:44:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 20:44:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:44:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:44:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:44:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:44:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:44:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:44:30 [DEBUG   ] Finished updating board flags in 1.615 seconds
2018-11-26 20:44:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 20:44:51 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:45:11 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:45:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:45:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:45:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:45:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:45:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:45:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:45:30 [DEBUG   ] Finished updating board flags in 1.635 seconds
2018-11-26 20:45:31 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 20:45:48 [DEBUG   ] Got command RPT from MCS: ref #636396
2018-11-26 20:45:48 [DEBUG   ] summary = ERROR
2018-11-26 20:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636396  13 58448 74748793 A  ERRORERROR'
2018-11-26 20:45:48 [DEBUG   ] Got command RPT from MCS: ref #636397
2018-11-26 20:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636397  77 58448 74748794 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:45:51 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:46:11 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 20:46:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:46:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:46:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:46:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:46:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:46:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:46:30 [DEBUG   ] Finished updating board flags in 1.652 seconds
2018-11-26 20:46:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 20:46:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 20:47:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 20:47:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:47:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 20:47:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:47:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:47:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:47:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:47:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:47:30 [DEBUG   ] Finished updating board flags in 1.529 seconds
2018-11-26 20:47:31 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2018-11-26 20:47:48 [DEBUG   ] Got command RPT from MCS: ref #636417
2018-11-26 20:47:48 [DEBUG   ] summary = ERROR
2018-11-26 20:47:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636417  13 58448 74868925 A  ERRORERROR'
2018-11-26 20:47:49 [DEBUG   ] Got command RPT from MCS: ref #636418
2018-11-26 20:47:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636418  77 58448 74869129 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:47:49 [DEBUG   ] Got command RPT from MCS: ref #636419
2018-11-26 20:47:49 [DEBUG   ] summary = ERROR
2018-11-26 20:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636419  13 58448 74869322 A  ERRORERROR'
2018-11-26 20:47:49 [DEBUG   ] Got command RPT from MCS: ref #636420
2018-11-26 20:47:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636420  77 58448 74869525 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:47:49 [DEBUG   ] Got command RPT from MCS: ref #636421
2018-11-26 20:47:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636421  12 58448 74869729 A  ERRORB•™š'
2018-11-26 20:47:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:48:11 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:48:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:48:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:48:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:48:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:48:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:48:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:48:30 [DEBUG   ] Finished updating board flags in 1.524 seconds
2018-11-26 20:48:31 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 20:48:51 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 20:49:11 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:49:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:49:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:49:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:49:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:49:30 [WARNING ] (13, 1, 'bot')
2018-11-26 20:49:30 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:49:30 [DEBUG   ] Finished updating board flags in 1.539 seconds
2018-11-26 20:49:31 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 20:49:52 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 20:50:12 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 20:50:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:50:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 20:50:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:50:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:50:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:50:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:50:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:50:31 [DEBUG   ] Finished updating board flags in 2.203 seconds
2018-11-26 20:50:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 20:50:48 [DEBUG   ] Got command RPT from MCS: ref #636426
2018-11-26 20:50:48 [DEBUG   ] summary = ERROR
2018-11-26 20:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636426  13 58448 75048801 A  ERRORERROR'
2018-11-26 20:50:48 [DEBUG   ] Got command RPT from MCS: ref #636427
2018-11-26 20:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636427  77 58448 75048802 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:50:52 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 20:51:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:51:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:51:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:51:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:51:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:51:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:51:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:51:31 [DEBUG   ] Finished updating board flags in 2.135 seconds
2018-11-26 20:51:32 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:51:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 20:52:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:52:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:52:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:52:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:52:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:52:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:52:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:52:31 [DEBUG   ] Finished updating board flags in 2.027 seconds
2018-11-26 20:52:32 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:52:49 [DEBUG   ] Got command RPT from MCS: ref #636449
2018-11-26 20:52:49 [DEBUG   ] summary = ERROR
2018-11-26 20:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636449  13 58448 75169009 A  ERRORERROR'
2018-11-26 20:52:49 [DEBUG   ] Got command RPT from MCS: ref #636450
2018-11-26 20:52:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636450  77 58448 75169213 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:52:49 [DEBUG   ] Got command RPT from MCS: ref #636451
2018-11-26 20:52:49 [DEBUG   ] summary = ERROR
2018-11-26 20:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636451  13 58448 75169405 A  ERRORERROR'
2018-11-26 20:52:49 [DEBUG   ] Got command RPT from MCS: ref #636453
2018-11-26 20:52:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636453  77 58448 75169609 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:52:49 [DEBUG   ] Got command RPT from MCS: ref #636454
2018-11-26 20:52:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:52:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636454  12 58448 75169813 A  ERRORB”™š'
2018-11-26 20:52:52 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 20:53:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 20:53:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:53:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:53:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:53:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:53:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:53:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:53:31 [DEBUG   ] Finished updating board flags in 2.056 seconds
2018-11-26 20:53:32 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 20:53:52 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 20:54:12 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 20:54:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:54:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 20:54:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:54:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:54:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:54:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:54:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:54:31 [DEBUG   ] Finished updating board flags in 1.941 seconds
2018-11-26 20:54:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 20:54:52 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 20:55:12 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 20:55:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:55:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:55:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:55:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:55:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:55:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:55:31 [DEBUG   ] Finished updating board flags in 1.797 seconds
2018-11-26 20:55:32 [DEBUG   ] Finished updating ant stats in 0.994 seconds
2018-11-26 20:55:48 [DEBUG   ] Got command RPT from MCS: ref #636463
2018-11-26 20:55:48 [DEBUG   ] summary = ERROR
2018-11-26 20:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636463  13 58448 75348801 A  ERRORERROR'
2018-11-26 20:55:48 [DEBUG   ] Got command RPT from MCS: ref #636464
2018-11-26 20:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636464  77 58448 75348802 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:55:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 20:56:12 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 20:56:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:56:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:56:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:56:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:56:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:56:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:56:31 [DEBUG   ] Finished updating board flags in 1.603 seconds
2018-11-26 20:56:32 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 20:56:52 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 20:57:12 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 20:57:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:57:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:57:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:57:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:57:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:57:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:57:31 [DEBUG   ] Finished updating board flags in 1.644 seconds
2018-11-26 20:57:32 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 20:57:49 [DEBUG   ] Got command RPT from MCS: ref #636480
2018-11-26 20:57:49 [DEBUG   ] summary = ERROR
2018-11-26 20:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636480  13 58448 75469093 A  ERRORERROR'
2018-11-26 20:57:49 [DEBUG   ] Got command RPT from MCS: ref #636481
2018-11-26 20:57:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636481  77 58448 75469297 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:57:49 [DEBUG   ] Got command RPT from MCS: ref #636482
2018-11-26 20:57:49 [DEBUG   ] summary = ERROR
2018-11-26 20:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636482  13 58448 75469501 A  ERRORERROR'
2018-11-26 20:57:49 [DEBUG   ] Got command RPT from MCS: ref #636483
2018-11-26 20:57:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 20:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636483  77 58448 75469693 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 20:57:49 [DEBUG   ] Got command RPT from MCS: ref #636484
2018-11-26 20:57:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 20:57:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636484  12 58448 75469897 A  ERRORB”™š'
2018-11-26 20:57:52 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 20:58:12 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 20:58:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:58:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:58:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:58:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:58:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:58:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:58:31 [DEBUG   ] Finished updating board flags in 1.613 seconds
2018-11-26 20:58:32 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:58:52 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 20:59:12 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 20:59:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 20:59:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:59:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:59:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:59:31 [WARNING ] (13, 1, 'bot')
2018-11-26 20:59:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 20:59:31 [DEBUG   ] Finished updating board flags in 1.501 seconds
2018-11-26 20:59:32 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 20:59:52 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 21:00:12 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 21:00:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:00:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:00:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:00:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:00:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:00:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:00:31 [DEBUG   ] Finished updating board flags in 1.402 seconds
2018-11-26 21:00:33 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 21:00:48 [DEBUG   ] Got command RPT from MCS: ref #636489
2018-11-26 21:00:48 [DEBUG   ] summary = ERROR
2018-11-26 21:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636489  13 58448 75648801 A  ERRORERROR'
2018-11-26 21:00:48 [DEBUG   ] Got command RPT from MCS: ref #636490
2018-11-26 21:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636490  77 58448 75648802 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:00:53 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 21:01:13 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 21:01:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:01:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:01:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:01:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:01:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:01:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:01:31 [DEBUG   ] Finished updating board flags in 1.392 seconds
2018-11-26 21:01:33 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 21:01:53 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 21:02:13 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 21:02:31 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:02:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:02:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:02:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:02:31 [WARNING ] (13, 1, 'bot')
2018-11-26 21:02:31 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:02:31 [DEBUG   ] Finished updating board flags in 1.357 seconds
2018-11-26 21:02:33 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 21:02:49 [DEBUG   ] Got command RPT from MCS: ref #636512
2018-11-26 21:02:49 [DEBUG   ] summary = ERROR
2018-11-26 21:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636512  13 58448 75769181 A  ERRORERROR'
2018-11-26 21:02:49 [DEBUG   ] Got command RPT from MCS: ref #636513
2018-11-26 21:02:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636513  77 58448 75769385 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:02:49 [DEBUG   ] Got command RPT from MCS: ref #636514
2018-11-26 21:02:49 [DEBUG   ] summary = ERROR
2018-11-26 21:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636514  13 58448 75769577 A  ERRORERROR'
2018-11-26 21:02:49 [DEBUG   ] Got command RPT from MCS: ref #636516
2018-11-26 21:02:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636516  77 58448 75769781 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:02:49 [DEBUG   ] Got command RPT from MCS: ref #636518
2018-11-26 21:02:49 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:02:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636518  12 58448 75769985 A  ERRORB”™š'
2018-11-26 21:02:53 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:03:13 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 21:03:31 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:03:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:03:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:03:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:03:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:03:32 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:03:32 [DEBUG   ] Finished updating board flags in 2.144 seconds
2018-11-26 21:03:33 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 21:03:53 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 21:04:13 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 21:04:31 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:04:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:04:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:04:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:04:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:04:32 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:04:32 [DEBUG   ] Finished updating board flags in 1.931 seconds
2018-11-26 21:04:33 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 21:04:53 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 21:05:13 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 21:05:31 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:05:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 21:05:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:05:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:05:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:05:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:05:32 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:05:32 [DEBUG   ] Finished updating board flags in 1.813 seconds
2018-11-26 21:05:34 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 21:05:48 [DEBUG   ] Got command RPT from MCS: ref #636526
2018-11-26 21:05:48 [DEBUG   ] summary = ERROR
2018-11-26 21:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636526  13 58448 75948805 A  ERRORERROR'
2018-11-26 21:05:48 [DEBUG   ] Got command RPT from MCS: ref #636527
2018-11-26 21:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636527  77 58448 75948806 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:05:54 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 21:06:14 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 21:06:31 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:06:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:06:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:06:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:06:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:06:32 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:06:32 [DEBUG   ] Finished updating board flags in 1.771 seconds
2018-11-26 21:06:34 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2018-11-26 21:06:54 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 21:07:14 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 21:07:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 21:07:31 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:07:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:07:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:07:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:07:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:07:32 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:07:32 [DEBUG   ] Finished updating board flags in 1.621 seconds
2018-11-26 21:07:34 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:07:49 [DEBUG   ] Got command RPT from MCS: ref #636547
2018-11-26 21:07:49 [DEBUG   ] summary = ERROR
2018-11-26 21:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636547  13 58448 76069269 A  ERRORERROR'
2018-11-26 21:07:49 [DEBUG   ] Got command RPT from MCS: ref #636548
2018-11-26 21:07:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636548  77 58448 76069473 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:07:49 [DEBUG   ] Got command RPT from MCS: ref #636549
2018-11-26 21:07:49 [DEBUG   ] summary = ERROR
2018-11-26 21:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636549  13 58448 76069666 A  ERRORERROR'
2018-11-26 21:07:49 [DEBUG   ] Got command RPT from MCS: ref #636550
2018-11-26 21:07:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:07:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636550  77 58448 76069869 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:07:50 [DEBUG   ] Got command RPT from MCS: ref #636551
2018-11-26 21:07:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:07:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636551  12 58448 76070073 A  ERRORB–™š'
2018-11-26 21:07:54 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 21:08:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:08:31 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:08:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:08:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:08:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:08:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:08:32 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:08:32 [DEBUG   ] Finished updating board flags in 1.586 seconds
2018-11-26 21:08:34 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:08:54 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:09:14 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 21:09:31 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:09:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:09:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:09:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:09:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:09:32 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:09:32 [DEBUG   ] Finished updating board flags in 1.439 seconds
2018-11-26 21:09:34 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 21:09:54 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 21:10:14 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 21:10:31 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:10:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:10:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:10:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:10:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:10:32 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:10:32 [DEBUG   ] Finished updating board flags in 1.370 seconds
2018-11-26 21:10:34 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 21:10:48 [DEBUG   ] Got command RPT from MCS: ref #636556
2018-11-26 21:10:48 [DEBUG   ] summary = ERROR
2018-11-26 21:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636556  13 58448 76248801 A  ERRORERROR'
2018-11-26 21:10:48 [DEBUG   ] Got command RPT from MCS: ref #636557
2018-11-26 21:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636557  77 58448 76248802 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:10:55 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 21:11:15 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 21:11:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:11:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:11:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:11:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:11:32 [WARNING ] (13, 1, 'bot')
2018-11-26 21:11:32 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:11:32 [DEBUG   ] Finished updating board flags in 1.371 seconds
2018-11-26 21:11:35 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 21:11:55 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 21:12:15 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:12:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 21:12:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:12:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:12:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:12:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:12:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:12:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:12:33 [DEBUG   ] Finished updating board flags in 2.034 seconds
2018-11-26 21:12:35 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 21:12:49 [DEBUG   ] Got command RPT from MCS: ref #636575
2018-11-26 21:12:49 [DEBUG   ] summary = ERROR
2018-11-26 21:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636575  13 58448 76369349 A  ERRORERROR'
2018-11-26 21:12:49 [DEBUG   ] Got command RPT from MCS: ref #636576
2018-11-26 21:12:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636576  77 58448 76369553 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:12:49 [DEBUG   ] Got command RPT from MCS: ref #636577
2018-11-26 21:12:49 [DEBUG   ] summary = ERROR
2018-11-26 21:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636577  13 58448 76369757 A  ERRORERROR'
2018-11-26 21:12:49 [DEBUG   ] Got command RPT from MCS: ref #636579
2018-11-26 21:12:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:12:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636579  77 58448 76369949 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:12:50 [DEBUG   ] Got command RPT from MCS: ref #636580
2018-11-26 21:12:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:12:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636580  12 58448 76370153 A  ERRORB—™š'
2018-11-26 21:12:55 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 21:13:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:13:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:13:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:13:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:13:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:13:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:13:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:13:33 [DEBUG   ] Finished updating board flags in 2.007 seconds
2018-11-26 21:13:35 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:13:55 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:14:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 21:14:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:14:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:14:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:14:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:14:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:14:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:14:33 [DEBUG   ] Finished updating board flags in 1.996 seconds
2018-11-26 21:14:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:14:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:15:15 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:15:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:15:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 21:15:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:15:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:15:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:15:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:15:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:15:33 [DEBUG   ] Finished updating board flags in 1.877 seconds
2018-11-26 21:15:35 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 21:15:48 [DEBUG   ] Got command RPT from MCS: ref #636589
2018-11-26 21:15:48 [DEBUG   ] summary = ERROR
2018-11-26 21:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636589  13 58448 76548801 A  ERRORERROR'
2018-11-26 21:15:48 [DEBUG   ] Got command RPT from MCS: ref #636590
2018-11-26 21:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636590  77 58448 76548802 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:15:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:16:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:16:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 21:16:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:16:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:16:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:16:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:16:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:16:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:16:33 [DEBUG   ] Finished updating board flags in 1.721 seconds
2018-11-26 21:16:35 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:16:55 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 21:17:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 21:17:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:17:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 21:17:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:17:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:17:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:17:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:17:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:17:33 [DEBUG   ] Finished updating board flags in 1.644 seconds
2018-11-26 21:17:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:17:49 [DEBUG   ] Got command RPT from MCS: ref #636610
2018-11-26 21:17:49 [DEBUG   ] summary = ERROR
2018-11-26 21:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636610  13 58448 76669441 A  ERRORERROR'
2018-11-26 21:17:49 [DEBUG   ] Got command RPT from MCS: ref #636611
2018-11-26 21:17:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636611  77 58448 76669633 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:17:49 [DEBUG   ] Got command RPT from MCS: ref #636612
2018-11-26 21:17:49 [DEBUG   ] summary = ERROR
2018-11-26 21:17:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636612  13 58448 76669837 A  ERRORERROR'
2018-11-26 21:17:50 [DEBUG   ] Got command RPT from MCS: ref #636613
2018-11-26 21:17:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:17:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636613  77 58448 76670041 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:17:50 [DEBUG   ] Got command RPT from MCS: ref #636614
2018-11-26 21:17:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:17:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636614  12 58448 76670245 A  ERRORB–™š'
2018-11-26 21:17:55 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:18:15 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 21:18:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:18:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:18:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:18:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:18:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:18:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:18:33 [DEBUG   ] Finished updating board flags in 1.574 seconds
2018-11-26 21:18:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 21:18:55 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 21:19:15 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 21:19:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:19:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:19:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:19:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:19:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:19:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:19:33 [DEBUG   ] Finished updating board flags in 1.404 seconds
2018-11-26 21:19:35 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 21:19:56 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 21:20:16 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 21:20:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:20:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:20:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:20:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:20:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:20:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:20:33 [DEBUG   ] Finished updating board flags in 1.407 seconds
2018-11-26 21:20:36 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 21:20:48 [DEBUG   ] Got command RPT from MCS: ref #636619
2018-11-26 21:20:48 [DEBUG   ] summary = ERROR
2018-11-26 21:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636619  13 58448 76848813 A  ERRORERROR'
2018-11-26 21:20:48 [DEBUG   ] Got command RPT from MCS: ref #636620
2018-11-26 21:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636620  77 58448 76848814 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:20:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 21:21:16 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:21:32 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:21:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:21:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:21:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:21:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:21:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:21:33 [DEBUG   ] Finished updating board flags in 1.391 seconds
2018-11-26 21:21:36 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 21:21:56 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 21:22:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 21:22:33 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:22:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:22:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:22:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:22:33 [WARNING ] (13, 1, 'bot')
2018-11-26 21:22:33 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:22:33 [DEBUG   ] Finished updating board flags in 1.367 seconds
2018-11-26 21:22:36 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 21:22:49 [DEBUG   ] Got command RPT from MCS: ref #636642
2018-11-26 21:22:49 [DEBUG   ] summary = ERROR
2018-11-26 21:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636642  13 58448 76969525 A  ERRORERROR'
2018-11-26 21:22:49 [DEBUG   ] Got command RPT from MCS: ref #636643
2018-11-26 21:22:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636643  77 58448 76969729 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:22:49 [DEBUG   ] Got command RPT from MCS: ref #636644
2018-11-26 21:22:49 [DEBUG   ] summary = ERROR
2018-11-26 21:22:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636644  13 58448 76969921 A  ERRORERROR'
2018-11-26 21:22:50 [DEBUG   ] Got command RPT from MCS: ref #636646
2018-11-26 21:22:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:22:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636646  77 58448 76970125 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:22:50 [DEBUG   ] Got command RPT from MCS: ref #636647
2018-11-26 21:22:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:22:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636647  12 58448 76970329 A  ERRORB–™š'
2018-11-26 21:22:56 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:23:16 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 21:23:33 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:23:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:23:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:23:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:23:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:23:34 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:23:34 [DEBUG   ] Finished updating board flags in 2.084 seconds
2018-11-26 21:23:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 21:23:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 21:24:16 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 21:24:33 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:24:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:24:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:24:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:24:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:24:34 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:24:34 [DEBUG   ] Finished updating board flags in 1.960 seconds
2018-11-26 21:24:36 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 21:24:57 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-26 21:25:17 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 21:25:33 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:25:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:25:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:25:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:25:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:25:34 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:25:34 [DEBUG   ] Finished updating board flags in 1.801 seconds
2018-11-26 21:25:37 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 21:25:48 [DEBUG   ] Got command RPT from MCS: ref #636656
2018-11-26 21:25:48 [DEBUG   ] summary = ERROR
2018-11-26 21:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636656  13 58448 77148813 A  ERRORERROR'
2018-11-26 21:25:48 [DEBUG   ] Got command RPT from MCS: ref #636657
2018-11-26 21:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636657  77 58448 77148814 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:25:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 21:26:17 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 21:26:33 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:26:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:26:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:26:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:26:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:26:34 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:26:34 [DEBUG   ] Finished updating board flags in 1.805 seconds
2018-11-26 21:26:37 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:26:57 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 21:27:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:27:33 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:27:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:27:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:27:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:27:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:27:34 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:27:34 [DEBUG   ] Finished updating board flags in 1.953 seconds
2018-11-26 21:27:37 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:27:49 [DEBUG   ] Got command RPT from MCS: ref #636673
2018-11-26 21:27:49 [DEBUG   ] summary = ERROR
2018-11-26 21:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636673  13 58448 77269605 A  ERRORERROR'
2018-11-26 21:27:49 [DEBUG   ] Got command RPT from MCS: ref #636674
2018-11-26 21:27:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:27:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636674  77 58448 77269809 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:27:50 [DEBUG   ] Got command RPT from MCS: ref #636675
2018-11-26 21:27:50 [DEBUG   ] summary = ERROR
2018-11-26 21:27:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636675  13 58448 77270013 A  ERRORERROR'
2018-11-26 21:27:50 [DEBUG   ] Got command RPT from MCS: ref #636676
2018-11-26 21:27:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:27:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636676  77 58448 77270205 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:27:50 [DEBUG   ] Got command RPT from MCS: ref #636677
2018-11-26 21:27:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:27:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636677  12 58448 77270409 A  ERRORB•™š'
2018-11-26 21:27:57 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 21:28:17 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 21:28:33 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:28:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:28:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:28:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:28:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:28:34 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:28:34 [DEBUG   ] Finished updating board flags in 1.652 seconds
2018-11-26 21:28:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 21:28:57 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 21:29:17 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 21:29:33 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:29:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:29:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:29:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:29:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:29:34 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:29:34 [DEBUG   ] Finished updating board flags in 1.589 seconds
2018-11-26 21:29:37 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 21:29:58 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 21:30:17 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 21:30:33 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:30:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:30:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:30:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:30:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:30:34 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:30:34 [DEBUG   ] Finished updating board flags in 1.439 seconds
2018-11-26 21:30:38 [DEBUG   ] Finished updating ant stats in 1.390 seconds
2018-11-26 21:30:48 [DEBUG   ] Got command RPT from MCS: ref #636682
2018-11-26 21:30:48 [DEBUG   ] summary = ERROR
2018-11-26 21:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636682  13 58448 77448809 A  ERRORERROR'
2018-11-26 21:30:48 [DEBUG   ] Got command RPT from MCS: ref #636683
2018-11-26 21:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636683  77 58448 77448810 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:30:58 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 21:31:18 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 21:31:33 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:31:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:31:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:31:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:31:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:31:34 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:31:34 [DEBUG   ] Finished updating board flags in 1.372 seconds
2018-11-26 21:31:38 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 21:31:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 21:32:18 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 21:32:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:32:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:32:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:32:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:32:34 [WARNING ] (13, 1, 'bot')
2018-11-26 21:32:34 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:32:34 [DEBUG   ] Finished updating board flags in 1.378 seconds
2018-11-26 21:32:38 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 21:32:49 [DEBUG   ] Got command RPT from MCS: ref #636705
2018-11-26 21:32:49 [DEBUG   ] summary = ERROR
2018-11-26 21:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636705  13 58448 77569693 A  ERRORERROR'
2018-11-26 21:32:49 [DEBUG   ] Got command RPT from MCS: ref #636706
2018-11-26 21:32:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:32:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636706  77 58448 77569897 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:32:50 [DEBUG   ] Got command RPT from MCS: ref #636707
2018-11-26 21:32:50 [DEBUG   ] summary = ERROR
2018-11-26 21:32:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636707  13 58448 77570101 A  ERRORERROR'
2018-11-26 21:32:50 [DEBUG   ] Got command RPT from MCS: ref #636709
2018-11-26 21:32:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:32:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636709  77 58448 77570301 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:32:50 [DEBUG   ] Got command RPT from MCS: ref #636710
2018-11-26 21:32:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:32:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636710  12 58448 77570501 A  ERRORB•™š'
2018-11-26 21:32:58 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 21:33:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:33:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:33:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:33:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:33:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:33:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:33:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:33:35 [DEBUG   ] Finished updating board flags in 2.035 seconds
2018-11-26 21:33:38 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 21:33:58 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:34:18 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 21:34:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:34:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:34:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:34:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:34:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:34:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:34:35 [DEBUG   ] Finished updating board flags in 1.997 seconds
2018-11-26 21:34:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 21:34:58 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 21:35:18 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 21:35:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:35:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 21:35:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:35:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:35:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:35:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:35:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:35:35 [DEBUG   ] Finished updating board flags in 1.815 seconds
2018-11-26 21:35:38 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 21:35:48 [DEBUG   ] Got command RPT from MCS: ref #636719
2018-11-26 21:35:48 [DEBUG   ] summary = ERROR
2018-11-26 21:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636719  13 58448 77748805 A  ERRORERROR'
2018-11-26 21:35:48 [DEBUG   ] Got command RPT from MCS: ref #636720
2018-11-26 21:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636720  77 58448 77748806 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:35:59 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 21:36:19 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 21:36:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:36:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:36:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:36:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:36:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:36:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:36:35 [DEBUG   ] Finished updating board flags in 1.809 seconds
2018-11-26 21:36:39 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 21:36:59 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 21:37:19 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 21:37:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:37:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:37:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:37:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:37:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:37:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:37:35 [DEBUG   ] Finished updating board flags in 1.770 seconds
2018-11-26 21:37:39 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:37:49 [DEBUG   ] Got command RPT from MCS: ref #636740
2018-11-26 21:37:49 [DEBUG   ] summary = ERROR
2018-11-26 21:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636740  13 58448 77869777 A  ERRORERROR'
2018-11-26 21:37:49 [DEBUG   ] Got command RPT from MCS: ref #636741
2018-11-26 21:37:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:37:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636741  77 58448 77869981 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:37:50 [DEBUG   ] Got command RPT from MCS: ref #636742
2018-11-26 21:37:50 [DEBUG   ] summary = ERROR
2018-11-26 21:37:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636742  13 58448 77870185 A  ERRORERROR'
2018-11-26 21:37:50 [DEBUG   ] Got command RPT from MCS: ref #636743
2018-11-26 21:37:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:37:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636743  77 58448 77870377 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:37:50 [DEBUG   ] Got command RPT from MCS: ref #636744
2018-11-26 21:37:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:37:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636744  12 58448 77870581 A  ERRORB”™š'
2018-11-26 21:37:59 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 21:38:19 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:38:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:38:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:38:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:38:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:38:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:38:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:38:35 [DEBUG   ] Finished updating board flags in 1.623 seconds
2018-11-26 21:38:39 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:38:59 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:39:19 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 21:39:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:39:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:39:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:39:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:39:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:39:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:39:35 [DEBUG   ] Finished updating board flags in 1.677 seconds
2018-11-26 21:39:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 21:39:59 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 21:40:19 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 21:40:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:40:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:40:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:40:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:40:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:40:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:40:35 [DEBUG   ] Finished updating board flags in 1.511 seconds
2018-11-26 21:40:39 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2018-11-26 21:40:48 [DEBUG   ] Got command RPT from MCS: ref #636749
2018-11-26 21:40:48 [DEBUG   ] summary = ERROR
2018-11-26 21:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636749  13 58448 78048822 A  ERRORERROR'
2018-11-26 21:40:48 [DEBUG   ] Got command RPT from MCS: ref #636750
2018-11-26 21:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636750  77 58448 78048822 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:40:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 21:41:19 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 21:41:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:41:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:41:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:41:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:41:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:41:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:41:35 [DEBUG   ] Finished updating board flags in 1.408 seconds
2018-11-26 21:41:40 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2018-11-26 21:42:00 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 21:42:20 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 21:42:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:42:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:42:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:42:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:42:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:42:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:42:35 [DEBUG   ] Finished updating board flags in 1.406 seconds
2018-11-26 21:42:40 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 21:42:49 [DEBUG   ] Got command RPT from MCS: ref #636768
2018-11-26 21:42:49 [DEBUG   ] summary = ERROR
2018-11-26 21:42:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636768  13 58448 78169865 A  ERRORERROR'
2018-11-26 21:42:50 [DEBUG   ] Got command RPT from MCS: ref #636769
2018-11-26 21:42:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:42:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636769  77 58448 78170057 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:42:50 [DEBUG   ] Got command RPT from MCS: ref #636770
2018-11-26 21:42:50 [DEBUG   ] summary = ERROR
2018-11-26 21:42:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636770  13 58448 78170261 A  ERRORERROR'
2018-11-26 21:42:50 [DEBUG   ] Got command RPT from MCS: ref #636771
2018-11-26 21:42:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:42:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636771  77 58448 78170465 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:42:50 [DEBUG   ] Got command RPT from MCS: ref #636773
2018-11-26 21:42:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:42:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636773  12 58448 78170669 A  ERRORB”™š'
2018-11-26 21:43:00 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 21:43:20 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 21:43:34 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:43:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:43:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:43:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:43:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:43:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:43:35 [DEBUG   ] Finished updating board flags in 1.396 seconds
2018-11-26 21:43:40 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:44:00 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 21:44:20 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 21:44:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:44:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:44:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:44:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:44:35 [WARNING ] (13, 1, 'bot')
2018-11-26 21:44:35 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:44:35 [DEBUG   ] Finished updating board flags in 1.387 seconds
2018-11-26 21:44:40 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:45:00 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:45:20 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 21:45:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:45:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 21:45:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:45:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:45:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:45:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:45:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:45:36 [DEBUG   ] Finished updating board flags in 2.038 seconds
2018-11-26 21:45:40 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 21:45:48 [DEBUG   ] Got command RPT from MCS: ref #636782
2018-11-26 21:45:48 [DEBUG   ] summary = ERROR
2018-11-26 21:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636782  13 58448 78348809 A  ERRORERROR'
2018-11-26 21:45:48 [DEBUG   ] Got command RPT from MCS: ref #636783
2018-11-26 21:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636783  77 58448 78348810 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:46:00 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 21:46:20 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 21:46:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:46:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:46:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:46:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:46:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:46:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:46:36 [DEBUG   ] Finished updating board flags in 1.980 seconds
2018-11-26 21:46:40 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 21:47:00 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 21:47:20 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 21:47:35 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 21:47:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:47:36 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 21:47:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:47:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:47:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:47:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:47:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:47:36 [DEBUG   ] Finished updating board flags in 1.809 seconds
2018-11-26 21:47:41 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 21:47:49 [DEBUG   ] Got command RPT from MCS: ref #636803
2018-11-26 21:47:49 [DEBUG   ] summary = ERROR
2018-11-26 21:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636803  13 58448 78469949 A  ERRORERROR'
2018-11-26 21:47:50 [DEBUG   ] Got command RPT from MCS: ref #636804
2018-11-26 21:47:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:47:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636804  77 58448 78470141 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:47:50 [DEBUG   ] Got command RPT from MCS: ref #636805
2018-11-26 21:47:50 [DEBUG   ] summary = ERROR
2018-11-26 21:47:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636805  13 58448 78470346 A  ERRORERROR'
2018-11-26 21:47:50 [DEBUG   ] Got command RPT from MCS: ref #636806
2018-11-26 21:47:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:47:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636806  77 58448 78470549 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:47:50 [DEBUG   ] Got command RPT from MCS: ref #636807
2018-11-26 21:47:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:47:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636807  12 58448 78470753 A  ERRORB•™š'
2018-11-26 21:48:00 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 21:48:20 [DEBUG   ] Finished updating ant stats in 0.399 seconds
2018-11-26 21:48:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:48:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:48:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:48:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:48:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:48:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:48:36 [DEBUG   ] Finished updating board flags in 1.812 seconds
2018-11-26 21:48:41 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 21:49:01 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 21:49:21 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 21:49:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:49:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:49:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:49:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:49:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:49:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:49:36 [DEBUG   ] Finished updating board flags in 1.811 seconds
2018-11-26 21:49:41 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 21:50:01 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:50:21 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 21:50:35 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-26 21:50:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:50:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:50:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:50:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:50:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:50:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:50:36 [DEBUG   ] Finished updating board flags in 1.777 seconds
2018-11-26 21:50:41 [DEBUG   ] Finished updating ant stats in 0.794 seconds
2018-11-26 21:50:48 [DEBUG   ] Got command RPT from MCS: ref #636812
2018-11-26 21:50:48 [DEBUG   ] summary = ERROR
2018-11-26 21:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636812  13 58448 78648813 A  ERRORERROR'
2018-11-26 21:50:48 [DEBUG   ] Got command RPT from MCS: ref #636813
2018-11-26 21:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636813  77 58448 78648814 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:51:01 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 21:51:21 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2018-11-26 21:51:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:51:36 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 21:51:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:51:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:51:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:51:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:51:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:51:36 [DEBUG   ] Finished updating board flags in 1.619 seconds
2018-11-26 21:51:42 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2018-11-26 21:52:02 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 21:52:22 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 21:52:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:52:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:52:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:52:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:52:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:52:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:52:36 [DEBUG   ] Finished updating board flags in 1.589 seconds
2018-11-26 21:52:42 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 21:52:50 [DEBUG   ] Got command RPT from MCS: ref #636835
2018-11-26 21:52:50 [DEBUG   ] summary = ERROR
2018-11-26 21:52:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636835  13 58448 78770033 A  ERRORERROR'
2018-11-26 21:52:50 [DEBUG   ] Got command RPT from MCS: ref #636836
2018-11-26 21:52:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:52:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636836  77 58448 78770237 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:52:50 [DEBUG   ] Got command RPT from MCS: ref #636837
2018-11-26 21:52:50 [DEBUG   ] summary = ERROR
2018-11-26 21:52:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636837  13 58448 78770429 A  ERRORERROR'
2018-11-26 21:52:50 [DEBUG   ] Got command RPT from MCS: ref #636839
2018-11-26 21:52:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:52:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636839  77 58448 78770633 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:52:50 [DEBUG   ] Got command RPT from MCS: ref #636840
2018-11-26 21:52:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:52:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636840  12 58448 78770837 A  ERRORB—™š'
2018-11-26 21:53:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 21:53:22 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 21:53:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:53:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:53:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:53:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:53:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:53:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:53:36 [DEBUG   ] Finished updating board flags in 1.410 seconds
2018-11-26 21:53:42 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:54:02 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 21:54:22 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:54:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:54:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 21:54:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:54:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:54:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:54:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:54:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:54:36 [DEBUG   ] Finished updating board flags in 1.393 seconds
2018-11-26 21:54:42 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 21:55:02 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-26 21:55:22 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 21:55:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 21:55:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:55:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 21:55:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:55:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:55:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:55:36 [WARNING ] (13, 1, 'bot')
2018-11-26 21:55:36 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:55:36 [DEBUG   ] Finished updating board flags in 1.358 seconds
2018-11-26 21:55:42 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 21:55:48 [DEBUG   ] Got command RPT from MCS: ref #636849
2018-11-26 21:55:48 [DEBUG   ] summary = ERROR
2018-11-26 21:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636849  13 58448 78948813 A  ERRORERROR'
2018-11-26 21:55:48 [DEBUG   ] Got command RPT from MCS: ref #636850
2018-11-26 21:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636850  77 58448 78948814 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:56:02 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 21:56:22 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 21:56:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:56:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:56:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:56:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:56:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:56:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:56:37 [DEBUG   ] Finished updating board flags in 2.059 seconds
2018-11-26 21:56:42 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 21:57:03 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 21:57:23 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 21:57:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:57:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:57:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:57:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:57:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:57:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:57:37 [DEBUG   ] Finished updating board flags in 1.993 seconds
2018-11-26 21:57:43 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 21:57:50 [DEBUG   ] Got command RPT from MCS: ref #636866
2018-11-26 21:57:50 [DEBUG   ] summary = ERROR
2018-11-26 21:57:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636866  13 58448 79070117 A  ERRORERROR'
2018-11-26 21:57:50 [DEBUG   ] Got command RPT from MCS: ref #636867
2018-11-26 21:57:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:57:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636867  77 58448 79070321 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:57:50 [DEBUG   ] Got command RPT from MCS: ref #636868
2018-11-26 21:57:50 [DEBUG   ] summary = ERROR
2018-11-26 21:57:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636868  13 58448 79070525 A  ERRORERROR'
2018-11-26 21:57:50 [DEBUG   ] Got command RPT from MCS: ref #636869
2018-11-26 21:57:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 21:57:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636869  77 58448 79070717 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 21:57:50 [DEBUG   ] Got command RPT from MCS: ref #636870
2018-11-26 21:57:50 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 21:57:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636870  12 58448 79070921 A  ERRORB—™š'
2018-11-26 21:58:03 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 21:58:23 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:58:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:58:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:58:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:58:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:58:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:58:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:58:37 [DEBUG   ] Finished updating board flags in 1.805 seconds
2018-11-26 21:58:43 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 21:59:03 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 21:59:23 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 21:59:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 21:59:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:59:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:59:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:59:37 [WARNING ] (13, 1, 'bot')
2018-11-26 21:59:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 21:59:37 [DEBUG   ] Finished updating board flags in 1.808 seconds
2018-11-26 21:59:43 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 22:00:03 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:00:23 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 22:00:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:00:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:00:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:00:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:00:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:00:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:00:37 [DEBUG   ] Finished updating board flags in 1.830 seconds
2018-11-26 22:00:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 22:00:48 [DEBUG   ] Got command RPT from MCS: ref #636875
2018-11-26 22:00:48 [DEBUG   ] summary = ERROR
2018-11-26 22:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636875  13 58448 79248817 A  ERRORERROR'
2018-11-26 22:00:48 [DEBUG   ] Got command RPT from MCS: ref #636876
2018-11-26 22:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636876  77 58448 79248818 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:01:03 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 22:01:23 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 22:01:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:01:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:01:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:01:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:01:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:01:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:01:37 [DEBUG   ] Finished updating board flags in 1.761 seconds
2018-11-26 22:01:43 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 22:02:03 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 22:02:23 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-26 22:02:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:02:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:02:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:02:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:02:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:02:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:02:37 [DEBUG   ] Finished updating board flags in 1.603 seconds
2018-11-26 22:02:44 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 22:02:50 [DEBUG   ] Got command RPT from MCS: ref #636898
2018-11-26 22:02:50 [DEBUG   ] summary = ERROR
2018-11-26 22:02:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636898  13 58448 79370201 A  ERRORERROR'
2018-11-26 22:02:50 [DEBUG   ] Got command RPT from MCS: ref #636899
2018-11-26 22:02:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:02:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636899  77 58448 79370405 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:02:50 [DEBUG   ] Got command RPT from MCS: ref #636900
2018-11-26 22:02:50 [DEBUG   ] summary = ERROR
2018-11-26 22:02:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636900  13 58448 79370597 A  ERRORERROR'
2018-11-26 22:02:50 [DEBUG   ] Got command RPT from MCS: ref #636901
2018-11-26 22:02:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:02:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636901  77 58448 79370801 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:02:51 [DEBUG   ] Got command RPT from MCS: ref #636903
2018-11-26 22:02:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:02:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636903  12 58448 79371005 A  ERRORB–™š'
2018-11-26 22:03:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 22:03:24 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 22:03:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:03:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:03:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:03:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:03:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:03:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:03:37 [DEBUG   ] Finished updating board flags in 1.595 seconds
2018-11-26 22:03:44 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 22:04:04 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 22:04:24 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 22:04:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:04:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:04:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:04:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:04:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:04:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:04:37 [DEBUG   ] Finished updating board flags in 1.411 seconds
2018-11-26 22:04:44 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 22:05:04 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:05:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 22:05:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:05:37 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 22:05:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:05:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:05:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:05:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:05:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:05:37 [DEBUG   ] Finished updating board flags in 1.409 seconds
2018-11-26 22:05:44 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 22:05:48 [DEBUG   ] Got command RPT from MCS: ref #636912
2018-11-26 22:05:48 [DEBUG   ] summary = ERROR
2018-11-26 22:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636912  13 58448 79548813 A  ERRORERROR'
2018-11-26 22:05:48 [DEBUG   ] Got command RPT from MCS: ref #636913
2018-11-26 22:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636913  77 58448 79548814 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:06:05 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 22:06:25 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 22:06:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:06:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:06:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:06:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:06:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:06:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:06:37 [DEBUG   ] Finished updating board flags in 1.408 seconds
2018-11-26 22:06:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 22:07:05 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 22:07:25 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 22:07:36 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:07:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:07:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:07:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:07:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:07:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:07:37 [DEBUG   ] Finished updating board flags in 1.380 seconds
2018-11-26 22:07:45 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 22:07:50 [DEBUG   ] Got command RPT from MCS: ref #636933
2018-11-26 22:07:50 [DEBUG   ] summary = ERROR
2018-11-26 22:07:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636933  13 58448 79670281 A  ERRORERROR'
2018-11-26 22:07:50 [DEBUG   ] Got command RPT from MCS: ref #636934
2018-11-26 22:07:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:07:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636934  77 58448 79670485 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:07:50 [DEBUG   ] Got command RPT from MCS: ref #636935
2018-11-26 22:07:50 [DEBUG   ] summary = ERROR
2018-11-26 22:07:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636935  13 58448 79670689 A  ERRORERROR'
2018-11-26 22:07:50 [DEBUG   ] Got command RPT from MCS: ref #636936
2018-11-26 22:07:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:07:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636936  77 58448 79670893 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:07:51 [DEBUG   ] Got command RPT from MCS: ref #636937
2018-11-26 22:07:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:07:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636937  12 58448 79671085 A  ERRORB•™š'
2018-11-26 22:08:05 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 22:08:25 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 22:08:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:08:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:08:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:08:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:08:37 [WARNING ] (13, 1, 'bot')
2018-11-26 22:08:37 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:08:37 [DEBUG   ] Finished updating board flags in 1.357 seconds
2018-11-26 22:08:45 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 22:09:05 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 22:09:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 22:09:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:09:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:09:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:09:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:09:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:09:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:09:38 [DEBUG   ] Finished updating board flags in 2.061 seconds
2018-11-26 22:09:45 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:10:05 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 22:10:25 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 22:10:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:10:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:10:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:10:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:10:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:10:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:10:38 [DEBUG   ] Finished updating board flags in 2.011 seconds
2018-11-26 22:10:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 22:10:48 [DEBUG   ] Got command RPT from MCS: ref #636942
2018-11-26 22:10:48 [DEBUG   ] summary = ERROR
2018-11-26 22:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636942  13 58448 79848821 A  ERRORERROR'
2018-11-26 22:10:48 [DEBUG   ] Got command RPT from MCS: ref #636943
2018-11-26 22:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636943  77 58448 79848822 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:11:05 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 22:11:25 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 22:11:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:11:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:11:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:11:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:11:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:11:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:11:38 [DEBUG   ] Finished updating board flags in 2.014 seconds
2018-11-26 22:11:45 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 22:12:06 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 22:12:25 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 22:12:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:12:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:12:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:12:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:12:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:12:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:12:38 [DEBUG   ] Finished updating board flags in 1.954 seconds
2018-11-26 22:12:46 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2018-11-26 22:12:50 [DEBUG   ] Got command RPT from MCS: ref #636961
2018-11-26 22:12:50 [DEBUG   ] summary = ERROR
2018-11-26 22:12:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636961  13 58448 79970373 A  ERRORERROR'
2018-11-26 22:12:50 [DEBUG   ] Got command RPT from MCS: ref #636962
2018-11-26 22:12:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:12:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636962  77 58448 79970577 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:12:50 [DEBUG   ] Got command RPT from MCS: ref #636963
2018-11-26 22:12:50 [DEBUG   ] summary = ERROR
2018-11-26 22:12:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636963  13 58448 79970769 A  ERRORERROR'
2018-11-26 22:12:50 [DEBUG   ] Got command RPT from MCS: ref #636965
2018-11-26 22:12:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:12:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636965  77 58448 79970973 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:12:51 [DEBUG   ] Got command RPT from MCS: ref #636966
2018-11-26 22:12:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:12:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636966  12 58448 79971177 A  ERRORB•™š'
2018-11-26 22:13:06 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 22:13:26 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 22:13:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:13:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:13:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:13:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:13:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:13:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:13:38 [DEBUG   ] Finished updating board flags in 1.822 seconds
2018-11-26 22:13:46 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 22:14:06 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 22:14:26 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 22:14:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:14:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:14:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:14:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:14:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:14:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:14:38 [DEBUG   ] Finished updating board flags in 1.782 seconds
2018-11-26 22:14:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 22:15:06 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2018-11-26 22:15:26 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 22:15:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:15:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:15:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:15:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:15:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:15:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:15:38 [DEBUG   ] Finished updating board flags in 1.612 seconds
2018-11-26 22:15:46 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 22:15:48 [DEBUG   ] Got command RPT from MCS: ref #636975
2018-11-26 22:15:48 [DEBUG   ] summary = ERROR
2018-11-26 22:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636975  13 58448 80148825 A  ERRORERROR'
2018-11-26 22:15:48 [DEBUG   ] Got command RPT from MCS: ref #636976
2018-11-26 22:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636976  77 58448 80148826 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:16:06 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-26 22:16:26 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 22:16:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:16:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:16:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:16:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:16:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:16:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:16:38 [DEBUG   ] Finished updating board flags in 1.604 seconds
2018-11-26 22:16:46 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 22:17:06 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 22:17:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 22:17:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:17:38 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 22:17:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:17:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:17:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:17:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:17:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:17:38 [DEBUG   ] Finished updating board flags in 1.597 seconds
2018-11-26 22:17:46 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 22:17:50 [DEBUG   ] Got command RPT from MCS: ref #636996
2018-11-26 22:17:50 [DEBUG   ] summary = ERROR
2018-11-26 22:17:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636996  13 58448 80270461 A  ERRORERROR'
2018-11-26 22:17:50 [DEBUG   ] Got command RPT from MCS: ref #636997
2018-11-26 22:17:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:17:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636997  77 58448 80270665 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:17:50 [DEBUG   ] Got command RPT from MCS: ref #636998
2018-11-26 22:17:50 [DEBUG   ] summary = ERROR
2018-11-26 22:17:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636998  13 58448 80270857 A  ERRORERROR'
2018-11-26 22:17:51 [DEBUG   ] Got command RPT from MCS: ref #636999
2018-11-26 22:17:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:17:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   636999  77 58448 80271061 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:17:51 [DEBUG   ] Got command RPT from MCS: ref #637000
2018-11-26 22:17:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:17:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637000  12 58448 80271265 A  ERRORB”™š'
2018-11-26 22:18:07 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 22:18:27 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 22:18:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:18:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:18:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:18:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:18:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:18:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:18:38 [DEBUG   ] Finished updating board flags in 1.403 seconds
2018-11-26 22:18:47 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 22:19:07 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 22:19:27 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 22:19:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:19:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:19:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:19:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:19:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:19:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:19:38 [DEBUG   ] Finished updating board flags in 1.409 seconds
2018-11-26 22:19:47 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 22:20:07 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 22:20:27 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 22:20:37 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:20:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:20:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:20:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:20:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:20:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:20:38 [DEBUG   ] Finished updating board flags in 1.386 seconds
2018-11-26 22:20:47 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:20:48 [DEBUG   ] Got command RPT from MCS: ref #637005
2018-11-26 22:20:48 [DEBUG   ] summary = ERROR
2018-11-26 22:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637005  13 58448 80448817 A  ERRORERROR'
2018-11-26 22:20:48 [DEBUG   ] Got command RPT from MCS: ref #637006
2018-11-26 22:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637006  77 58448 80448818 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:21:07 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 22:21:27 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 22:21:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:21:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:21:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:21:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:21:38 [WARNING ] (13, 1, 'bot')
2018-11-26 22:21:38 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:21:38 [DEBUG   ] Finished updating board flags in 1.376 seconds
2018-11-26 22:21:47 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 22:22:07 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 22:22:28 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2018-11-26 22:22:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:22:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:22:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:22:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:22:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:22:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:22:39 [DEBUG   ] Finished updating board flags in 2.059 seconds
2018-11-26 22:22:48 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 22:22:50 [DEBUG   ] Got command RPT from MCS: ref #637028
2018-11-26 22:22:50 [DEBUG   ] summary = ERROR
2018-11-26 22:22:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637028  13 58448 80570541 A  ERRORERROR'
2018-11-26 22:22:50 [DEBUG   ] Got command RPT from MCS: ref #637029
2018-11-26 22:22:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:22:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637029  77 58448 80570745 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:22:50 [DEBUG   ] Got command RPT from MCS: ref #637030
2018-11-26 22:22:50 [DEBUG   ] summary = ERROR
2018-11-26 22:22:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637030  13 58448 80570949 A  ERRORERROR'
2018-11-26 22:22:51 [DEBUG   ] Got command RPT from MCS: ref #637032
2018-11-26 22:22:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:22:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637032  77 58448 80571141 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:22:51 [DEBUG   ] Got command RPT from MCS: ref #637033
2018-11-26 22:22:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:22:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637033  12 58448 80571345 A  ERRORB”™š'
2018-11-26 22:23:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 22:23:28 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 22:23:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:23:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:23:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:23:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:23:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:23:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:23:39 [DEBUG   ] Finished updating board flags in 2.009 seconds
2018-11-26 22:23:48 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 22:24:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 22:24:28 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 22:24:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:24:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:24:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:24:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:24:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:24:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:24:39 [DEBUG   ] Finished updating board flags in 1.961 seconds
2018-11-26 22:24:48 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 22:25:08 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 22:25:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 22:25:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:25:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:25:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:25:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:25:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:25:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:25:39 [DEBUG   ] Finished updating board flags in 1.829 seconds
2018-11-26 22:25:48 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:25:48 [DEBUG   ] Got command RPT from MCS: ref #637042
2018-11-26 22:25:48 [DEBUG   ] summary = ERROR
2018-11-26 22:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637042  13 58448 80748825 A  ERRORERROR'
2018-11-26 22:25:48 [DEBUG   ] Got command RPT from MCS: ref #637043
2018-11-26 22:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637043  77 58448 80748826 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:26:08 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 22:26:28 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 22:26:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:26:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:26:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:26:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:26:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:26:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:26:39 [DEBUG   ] Finished updating board flags in 1.815 seconds
2018-11-26 22:26:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 22:27:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 22:27:28 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 22:27:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:27:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:27:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:27:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:27:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:27:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:27:39 [DEBUG   ] Finished updating board flags in 1.810 seconds
2018-11-26 22:27:48 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 22:27:50 [DEBUG   ] Got command RPT from MCS: ref #637059
2018-11-26 22:27:50 [DEBUG   ] summary = ERROR
2018-11-26 22:27:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637059  13 58448 80870637 A  ERRORERROR'
2018-11-26 22:27:50 [DEBUG   ] Got command RPT from MCS: ref #637060
2018-11-26 22:27:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:27:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637060  77 58448 80870829 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:27:51 [DEBUG   ] Got command RPT from MCS: ref #637061
2018-11-26 22:27:51 [DEBUG   ] summary = ERROR
2018-11-26 22:27:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637061  13 58448 80871033 A  ERRORERROR'
2018-11-26 22:27:51 [DEBUG   ] Got command RPT from MCS: ref #637062
2018-11-26 22:27:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:27:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637062  77 58448 80871237 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:27:51 [DEBUG   ] Got command RPT from MCS: ref #637063
2018-11-26 22:27:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:27:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637063  12 58448 80871429 A  ERRORB”™š'
2018-11-26 22:28:09 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 22:28:29 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 22:28:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:28:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:28:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:28:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:28:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:28:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:28:39 [DEBUG   ] Finished updating board flags in 1.761 seconds
2018-11-26 22:28:49 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 22:29:09 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 22:29:29 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 22:29:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:29:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:29:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:29:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:29:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:29:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:29:39 [DEBUG   ] Finished updating board flags in 1.603 seconds
2018-11-26 22:29:49 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 22:30:09 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 22:30:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 22:30:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:30:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:30:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:30:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:30:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:30:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:30:39 [DEBUG   ] Finished updating board flags in 1.611 seconds
2018-11-26 22:30:48 [DEBUG   ] Got command RPT from MCS: ref #637068
2018-11-26 22:30:48 [DEBUG   ] summary = ERROR
2018-11-26 22:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637068  13 58448 81048825 A  ERRORERROR'
2018-11-26 22:30:48 [DEBUG   ] Got command RPT from MCS: ref #637069
2018-11-26 22:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637069  77 58448 81048826 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:30:49 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:31:09 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:31:29 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 22:31:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:31:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:31:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:31:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:31:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:31:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:31:39 [DEBUG   ] Finished updating board flags in 1.618 seconds
2018-11-26 22:31:49 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 22:32:09 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 22:32:11 [DEBUG   ] Got command TBW from MCS: ref #637090
2018-11-26 22:32:11 [DEBUG   ] tbw = exited with status True, code 0x00
2018-11-26 22:32:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBW   637090   8 58448 81131370 A  ERROR'
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 pkill tbn
2018-11-26 22:32:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 pkill tbn
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa27', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '1', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa24', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '2', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa23', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '3', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa18', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '4', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa10', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '5', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa32', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '6', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa11', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '7', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa30', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '8', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa12', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '9', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa8', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '10', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa13', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '11', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa26', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '12', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa14', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '13', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa22', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '14', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa20', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '15', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa1', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '16', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa5', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '17', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa25', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '18', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa35', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '19', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa19', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '20', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa21', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '21', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa6', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '22', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa7', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '23', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa33', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '24', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa15', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '25', '12000000']
2018-11-26 22:32:14 [DEBUG   ] spawn_process: at 2018-11-26 22:32:14 running ['/usr/local/bin/lwa28', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '26', '12000000']
2018-11-26 22:32:14 [INFO    ] Finished spawning TBW
2018-11-26 22:32:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:32:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:32:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:32:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:32:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:32:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:32:39 [DEBUG   ] Finished updating board flags in 1.428 seconds
2018-11-26 22:32:50 [DEBUG   ] Got command RPT from MCS: ref #637095
2018-11-26 22:32:50 [DEBUG   ] summary = ERROR
2018-11-26 22:32:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637095  13 58448 81170717 A  ERRORERROR'
2018-11-26 22:32:50 [DEBUG   ] Got command RPT from MCS: ref #637096
2018-11-26 22:32:50 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:32:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637096  77 58448 81170921 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:32:51 [DEBUG   ] Got command RPT from MCS: ref #637097
2018-11-26 22:32:51 [DEBUG   ] summary = ERROR
2018-11-26 22:32:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637097  13 58448 81171125 A  ERRORERROR'
2018-11-26 22:32:51 [DEBUG   ] Got command RPT from MCS: ref #637099
2018-11-26 22:32:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:32:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637099  77 58448 81171317 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:32:51 [DEBUG   ] Got command RPT from MCS: ref #637100
2018-11-26 22:32:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:32:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637100  12 58448 81171521 A  ERRORB”™š'
2018-11-26 22:33:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:33:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:33:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:33:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:33:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:33:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:33:39 [DEBUG   ] Finished updating board flags in 1.514 seconds
2018-11-26 22:34:02 [INFO    ] __waitForTBW - TBW finished in 108.055 s
2018-11-26 22:34:03 [DEBUG   ] Finished updating ant stats in 0.344 seconds
2018-11-26 22:34:03 [INFO    ] Started antennas statistics background thread
2018-11-26 22:34:03 [INFO    ] Resuming TBN at 38.100 MHz after TBW acquisition
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '1', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '2', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '3', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '4', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '5', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '6', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '7', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '8', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '9', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '10', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '11', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '12', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '13', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '14', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '15', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '16', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '17', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '18', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '19', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '20', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '21', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '22', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '23', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '24', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '25', '1543271645']
2018-11-26 22:34:03 [DEBUG   ] spawn_process: at 2018-11-26 22:34:03 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '26', '1543271645']
2018-11-26 22:34:03 [INFO    ] Finished spawning TBN in 0.050 s
2018-11-26 22:34:24 [DEBUG   ] Finished updating ant stats in 1.271 seconds
2018-11-26 22:34:38 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:34:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:34:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:34:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:34:39 [WARNING ] (13, 1, 'bot')
2018-11-26 22:34:39 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:34:39 [DEBUG   ] Finished updating board flags in 1.647 seconds
2018-11-26 22:34:44 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 22:35:04 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 22:35:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 22:35:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:35:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:35:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:35:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:35:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:35:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:35:40 [DEBUG   ] Finished updating board flags in 1.776 seconds
2018-11-26 22:35:44 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 22:35:48 [DEBUG   ] Got command RPT from MCS: ref #637114
2018-11-26 22:35:48 [DEBUG   ] summary = ERROR
2018-11-26 22:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637114  13 58448 81348825 A  ERRORERROR'
2018-11-26 22:35:48 [DEBUG   ] Got command RPT from MCS: ref #637115
2018-11-26 22:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637115  77 58448 81348826 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:36:04 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 22:36:24 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 22:36:33 [DEBUG   ] Got command DRX from MCS: ref #637135
2018-11-26 22:36:33 [INFO    ] sending drx command to lwa34
2018-11-26 22:36:33 [DEBUG   ] run_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 1 35100000.0 7 9 0']
2018-11-26 22:36:33 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-26 22:36:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   637135   8 58448 81393013 A  ERROR'
2018-11-26 22:36:33 [DEBUG   ] Got command DRX from MCS: ref #637136
2018-11-26 22:36:33 [INFO    ] sending drx command to lwa16
2018-11-26 22:36:33 [DEBUG   ] run_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 64500000.0 7 9 0']
2018-11-26 22:36:33 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-26 22:36:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   637136   8 58448 81393017 A  ERROR'
2018-11-26 22:36:33 [DEBUG   ] Got command DRX from MCS: ref #637137
2018-11-26 22:36:33 [INFO    ] sending drx command to lwa34
2018-11-26 22:36:33 [DEBUG   ] run_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 2 49800000.0 7 9 0']
2018-11-26 22:36:33 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-26 22:36:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   637137   8 58448 81393021 A  ERROR'
2018-11-26 22:36:33 [DEBUG   ] Got command DRX from MCS: ref #637138
2018-11-26 22:36:33 [INFO    ] sending drx command to lwa16
2018-11-26 22:36:33 [DEBUG   ] run_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 79200000.0 7 9 0']
2018-11-26 22:36:33 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-26 22:36:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   637138   8 58448 81393025 A  ERROR'
2018-11-26 22:36:33 [DEBUG   ] Got command BAM from MCS: ref #637140
2018-11-26 22:36:33 [INFO    ] Finished processing BAM request
2018-11-26 22:36:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:36:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637140   8 58448 81393029 A  ERROR'
2018-11-26 22:36:33 [DEBUG   ] Got command BAM from MCS: ref #637139
2018-11-26 22:36:33 [INFO    ] Finished processing BAM request
2018-11-26 22:36:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:36:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637139   8 58448 81393033 A  ERROR'
2018-11-26 22:36:33 [INFO    ] Finished starting DRX
2018-11-26 22:36:33 [INFO    ] Finished starting DRX
2018-11-26 22:36:33 [INFO    ] Finished starting DRX
2018-11-26 22:36:33 [INFO    ] Finished starting DRX
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 22:36:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 22:36:33 [DEBUG   ] spawn_process: at 2018-11-26 22:36:33 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 22:36:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:36:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2018-11-26 22:36:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:36:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:36:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:36:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:36:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:36:40 [DEBUG   ] Finished updating board flags in 1.636 seconds
2018-11-26 22:36:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 22:37:04 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:37:14 [DEBUG   ] Got command BAM from MCS: ref #637147
2018-11-26 22:37:14 [INFO    ] Finished processing BAM request
2018-11-26 22:37:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:37:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637147   8 58448 81434033 A  ERROR'
2018-11-26 22:37:14 [DEBUG   ] Got command BAM from MCS: ref #637148
2018-11-26 22:37:14 [INFO    ] Finished processing BAM request
2018-11-26 22:37:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:37:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637148   8 58448 81434037 A  ERROR'
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 22:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 22:37:14 [DEBUG   ] spawn_process: at 2018-11-26 22:37:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 22:37:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 22:37:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:37:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:37:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:37:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:37:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:37:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:37:40 [DEBUG   ] Finished updating board flags in 2.331 seconds
2018-11-26 22:37:44 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 22:37:50 [DEBUG   ] Got command RPT from MCS: ref #637149
2018-11-26 22:37:50 [DEBUG   ] summary = ERROR
2018-11-26 22:37:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637149  13 58448 81470801 A  ERRORERROR'
2018-11-26 22:37:51 [DEBUG   ] Got command RPT from MCS: ref #637150
2018-11-26 22:37:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:37:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637150  77 58448 81471005 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:37:51 [DEBUG   ] Got command RPT from MCS: ref #637151
2018-11-26 22:37:51 [DEBUG   ] summary = ERROR
2018-11-26 22:37:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637151  13 58448 81471209 A  ERRORERROR'
2018-11-26 22:37:51 [DEBUG   ] Got command RPT from MCS: ref #637152
2018-11-26 22:37:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:37:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637152  77 58448 81471413 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:37:51 [DEBUG   ] Got command RPT from MCS: ref #637153
2018-11-26 22:37:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:37:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637153  12 58448 81471605 A  ERRORB™™š'
2018-11-26 22:37:55 [DEBUG   ] Got command BAM from MCS: ref #637154
2018-11-26 22:37:55 [INFO    ] Finished processing BAM request
2018-11-26 22:37:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:37:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637154   8 58448 81475029 A  ERROR'
2018-11-26 22:37:55 [DEBUG   ] Got command BAM from MCS: ref #637155
2018-11-26 22:37:55 [INFO    ] Finished processing BAM request
2018-11-26 22:37:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:37:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637155   8 58448 81475033 A  ERROR'
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 22:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 22:37:55 [DEBUG   ] spawn_process: at 2018-11-26 22:37:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 22:38:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 22:38:24 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 22:38:36 [DEBUG   ] Got command BAM from MCS: ref #637156
2018-11-26 22:38:36 [INFO    ] Finished processing BAM request
2018-11-26 22:38:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:38:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637156   8 58448 81516033 A  ERROR'
2018-11-26 22:38:36 [DEBUG   ] Got command BAM from MCS: ref #637157
2018-11-26 22:38:36 [INFO    ] Finished processing BAM request
2018-11-26 22:38:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:38:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637157   8 58448 81516037 A  ERROR'
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 22:38:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 22:38:36 [DEBUG   ] spawn_process: at 2018-11-26 22:38:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 22:38:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 22:38:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:38:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:38:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:38:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:38:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:38:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:38:40 [DEBUG   ] Finished updating board flags in 2.052 seconds
2018-11-26 22:38:44 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 22:39:04 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 22:39:17 [DEBUG   ] Got command BAM from MCS: ref #637158
2018-11-26 22:39:17 [INFO    ] Finished processing BAM request
2018-11-26 22:39:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:39:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637158   8 58448 81557029 A  ERROR'
2018-11-26 22:39:17 [DEBUG   ] Got command BAM from MCS: ref #637159
2018-11-26 22:39:17 [INFO    ] Finished processing BAM request
2018-11-26 22:39:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:39:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637159   8 58448 81557033 A  ERROR'
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 22:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 22:39:17 [DEBUG   ] spawn_process: at 2018-11-26 22:39:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 22:39:24 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 22:39:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-26 22:39:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:39:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 22:39:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:39:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:39:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:39:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:39:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:39:40 [DEBUG   ] Finished updating board flags in 2.006 seconds
2018-11-26 22:39:45 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 22:39:58 [DEBUG   ] Got command BAM from MCS: ref #637161
2018-11-26 22:39:58 [INFO    ] Finished processing BAM request
2018-11-26 22:39:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637161   8 58448 81598029 A  ERROR'
2018-11-26 22:39:58 [DEBUG   ] Got command BAM from MCS: ref #637160
2018-11-26 22:39:58 [INFO    ] Finished processing BAM request
2018-11-26 22:39:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637160   8 58448 81598033 A  ERROR'
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 22:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 22:39:58 [DEBUG   ] spawn_process: at 2018-11-26 22:39:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 22:40:04 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 22:40:25 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2018-11-26 22:40:39 [DEBUG   ] Got command BAM from MCS: ref #637167
2018-11-26 22:40:39 [INFO    ] Finished processing BAM request
2018-11-26 22:40:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:40:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637167   8 58448 81639033 A  ERROR'
2018-11-26 22:40:39 [DEBUG   ] Got command BAM from MCS: ref #637166
2018-11-26 22:40:39 [INFO    ] Finished processing BAM request
2018-11-26 22:40:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:40:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637166   8 58448 81639041 A  ERROR'
2018-11-26 22:40:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 22:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 22:40:39 [DEBUG   ] spawn_process: at 2018-11-26 22:40:39 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 22:40:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:40:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:40:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:40:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:40:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:40:40 [DEBUG   ] Finished updating board flags in 1.995 seconds
2018-11-26 22:40:45 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 22:40:48 [DEBUG   ] Got command RPT from MCS: ref #637168
2018-11-26 22:40:48 [DEBUG   ] summary = ERROR
2018-11-26 22:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637168  13 58448 81648829 A  ERRORERROR'
2018-11-26 22:40:48 [DEBUG   ] Got command RPT from MCS: ref #637169
2018-11-26 22:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637169  77 58448 81648830 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:41:05 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 22:41:20 [DEBUG   ] Got command BAM from MCS: ref #637186
2018-11-26 22:41:20 [INFO    ] Finished processing BAM request
2018-11-26 22:41:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:41:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637186   8 58448 81680025 A  ERROR'
2018-11-26 22:41:20 [DEBUG   ] Got command BAM from MCS: ref #637185
2018-11-26 22:41:20 [INFO    ] Finished processing BAM request
2018-11-26 22:41:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:41:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637185   8 58448 81680029 A  ERROR'
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 22:41:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 22:41:20 [DEBUG   ] spawn_process: at 2018-11-26 22:41:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 22:41:25 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 22:41:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:41:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:41:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:41:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:41:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:41:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:41:40 [DEBUG   ] Finished updating board flags in 1.818 seconds
2018-11-26 22:41:45 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 22:42:01 [DEBUG   ] Got command BAM from MCS: ref #637187
2018-11-26 22:42:01 [INFO    ] Finished processing BAM request
2018-11-26 22:42:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:42:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637187   8 58448 81721033 A  ERROR'
2018-11-26 22:42:01 [DEBUG   ] Got command BAM from MCS: ref #637188
2018-11-26 22:42:01 [INFO    ] Finished processing BAM request
2018-11-26 22:42:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:42:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637188   8 58448 81721037 A  ERROR'
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 22:42:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 22:42:01 [DEBUG   ] spawn_process: at 2018-11-26 22:42:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 22:42:05 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 22:42:25 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 22:42:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:42:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:42:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:42:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:42:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:42:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:42:40 [DEBUG   ] Finished updating board flags in 1.789 seconds
2018-11-26 22:42:42 [DEBUG   ] Got command BAM from MCS: ref #637192
2018-11-26 22:42:42 [INFO    ] Finished processing BAM request
2018-11-26 22:42:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:42:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637192   8 58448 81762025 A  ERROR'
2018-11-26 22:42:42 [DEBUG   ] Got command BAM from MCS: ref #637191
2018-11-26 22:42:42 [INFO    ] Finished processing BAM request
2018-11-26 22:42:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:42:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637191   8 58448 81762029 A  ERROR'
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 22:42:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 22:42:42 [DEBUG   ] spawn_process: at 2018-11-26 22:42:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 22:42:45 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 22:42:50 [DEBUG   ] Got command RPT from MCS: ref #637193
2018-11-26 22:42:50 [DEBUG   ] summary = ERROR
2018-11-26 22:42:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637193  13 58448 81770893 A  ERRORERROR'
2018-11-26 22:42:51 [DEBUG   ] Got command RPT from MCS: ref #637194
2018-11-26 22:42:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:42:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637194  77 58448 81771097 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:42:51 [DEBUG   ] Got command RPT from MCS: ref #637195
2018-11-26 22:42:51 [DEBUG   ] summary = ERROR
2018-11-26 22:42:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637195  13 58448 81771289 A  ERRORERROR'
2018-11-26 22:42:51 [DEBUG   ] Got command RPT from MCS: ref #637197
2018-11-26 22:42:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:42:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637197  77 58448 81771493 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:42:51 [DEBUG   ] Got command RPT from MCS: ref #637198
2018-11-26 22:42:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:42:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637198  12 58448 81771697 A  ERRORB�ff'
2018-11-26 22:43:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 22:43:23 [DEBUG   ] Got command BAM from MCS: ref #637208
2018-11-26 22:43:23 [INFO    ] Finished processing BAM request
2018-11-26 22:43:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:43:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637208   8 58448 81803029 A  ERROR'
2018-11-26 22:43:23 [DEBUG   ] Got command BAM from MCS: ref #637207
2018-11-26 22:43:23 [INFO    ] Finished processing BAM request
2018-11-26 22:43:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:43:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637207   8 58448 81803033 A  ERROR'
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 22:43:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 22:43:23 [DEBUG   ] spawn_process: at 2018-11-26 22:43:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 22:43:25 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:43:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 22:43:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-26 22:43:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:43:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-26 22:43:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-26 22:43:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:43:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:43:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:43:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:43:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:43:40 [DEBUG   ] Finished updating board flags in 1.606 seconds
2018-11-26 22:43:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 22:44:04 [DEBUG   ] Got command BAM from MCS: ref #637209
2018-11-26 22:44:04 [INFO    ] Finished processing BAM request
2018-11-26 22:44:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:44:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637209   8 58448 81844029 A  ERROR'
2018-11-26 22:44:04 [DEBUG   ] Got command BAM from MCS: ref #637210
2018-11-26 22:44:04 [INFO    ] Finished processing BAM request
2018-11-26 22:44:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:44:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637210   8 58448 81844033 A  ERROR'
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 22:44:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 22:44:04 [DEBUG   ] spawn_process: at 2018-11-26 22:44:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 22:44:05 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 22:44:25 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 22:44:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 22:44:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:44:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:44:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:44:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:44:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:44:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:44:40 [DEBUG   ] Finished updating board flags in 1.610 seconds
2018-11-26 22:44:45 [DEBUG   ] Got command BAM from MCS: ref #637212
2018-11-26 22:44:45 [INFO    ] Finished processing BAM request
2018-11-26 22:44:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:44:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637212   8 58448 81885029 A  ERROR'
2018-11-26 22:44:45 [DEBUG   ] Got command BAM from MCS: ref #637211
2018-11-26 22:44:45 [INFO    ] Finished processing BAM request
2018-11-26 22:44:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:44:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637211   8 58448 81885033 A  ERROR'
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 22:44:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 22:44:45 [DEBUG   ] spawn_process: at 2018-11-26 22:44:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 22:44:46 [DEBUG   ] Finished updating ant stats in 1.604 seconds
2018-11-26 22:45:05 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 22:45:26 [DEBUG   ] Got command BAM from MCS: ref #637213
2018-11-26 22:45:26 [INFO    ] Finished processing BAM request
2018-11-26 22:45:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:45:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637213   8 58448 81926033 A  ERROR'
2018-11-26 22:45:26 [DEBUG   ] Got command BAM from MCS: ref #637214
2018-11-26 22:45:26 [INFO    ] Finished processing BAM request
2018-11-26 22:45:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:45:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637214   8 58448 81926037 A  ERROR'
2018-11-26 22:45:26 [DEBUG   ] Finished updating ant stats in 1.599 seconds
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 22:45:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 22:45:26 [DEBUG   ] spawn_process: at 2018-11-26 22:45:26 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 22:45:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:45:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:45:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:45:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:45:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:45:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:45:40 [DEBUG   ] Finished updating board flags in 1.586 seconds
2018-11-26 22:45:45 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 22:45:48 [DEBUG   ] Got command RPT from MCS: ref #637215
2018-11-26 22:45:48 [DEBUG   ] summary = ERROR
2018-11-26 22:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637215  13 58448 81948825 A  ERRORERROR'
2018-11-26 22:45:48 [DEBUG   ] Got command RPT from MCS: ref #637216
2018-11-26 22:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637216  77 58448 81948826 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:46:06 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 22:46:07 [DEBUG   ] Got command BAM from MCS: ref #637237
2018-11-26 22:46:07 [INFO    ] Finished processing BAM request
2018-11-26 22:46:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:46:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637237   8 58448 81967033 A  ERROR'
2018-11-26 22:46:07 [DEBUG   ] Got command BAM from MCS: ref #637236
2018-11-26 22:46:07 [INFO    ] Finished processing BAM request
2018-11-26 22:46:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:46:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637236   8 58448 81967037 A  ERROR'
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 22:46:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 22:46:07 [DEBUG   ] spawn_process: at 2018-11-26 22:46:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 22:46:26 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 22:46:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:46:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:46:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:46:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:46:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:46:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:46:40 [DEBUG   ] Finished updating board flags in 1.453 seconds
2018-11-26 22:46:46 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 22:46:48 [DEBUG   ] Got command BAM from MCS: ref #637238
2018-11-26 22:46:48 [INFO    ] Finished processing BAM request
2018-11-26 22:46:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:46:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637238   8 58448 82008025 A  ERROR'
2018-11-26 22:46:48 [DEBUG   ] Got command BAM from MCS: ref #637239
2018-11-26 22:46:48 [INFO    ] Finished processing BAM request
2018-11-26 22:46:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:46:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637239   8 58448 82008029 A  ERROR'
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 22:46:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 22:46:48 [DEBUG   ] spawn_process: at 2018-11-26 22:46:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 22:47:06 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 22:47:26 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 22:47:29 [DEBUG   ] Got command BAM from MCS: ref #637241
2018-11-26 22:47:29 [INFO    ] Finished processing BAM request
2018-11-26 22:47:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:47:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637241   8 58448 82049025 A  ERROR'
2018-11-26 22:47:29 [DEBUG   ] Got command BAM from MCS: ref #637240
2018-11-26 22:47:29 [INFO    ] Finished processing BAM request
2018-11-26 22:47:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:47:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637240   8 58448 82049029 A  ERROR'
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 22:47:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 22:47:29 [DEBUG   ] spawn_process: at 2018-11-26 22:47:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 22:47:39 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:47:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:47:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:47:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:47:40 [WARNING ] (13, 1, 'bot')
2018-11-26 22:47:40 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:47:40 [DEBUG   ] Finished updating board flags in 1.393 seconds
2018-11-26 22:47:46 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 22:47:50 [DEBUG   ] Got command RPT from MCS: ref #637242
2018-11-26 22:47:50 [DEBUG   ] summary = ERROR
2018-11-26 22:47:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637242  13 58448 82070977 A  ERRORERROR'
2018-11-26 22:47:51 [DEBUG   ] Got command RPT from MCS: ref #637243
2018-11-26 22:47:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:47:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637243  77 58448 82071181 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:47:51 [DEBUG   ] Got command RPT from MCS: ref #637244
2018-11-26 22:47:51 [DEBUG   ] summary = ERROR
2018-11-26 22:47:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637244  13 58448 82071385 A  ERRORERROR'
2018-11-26 22:47:51 [DEBUG   ] Got command RPT from MCS: ref #637245
2018-11-26 22:47:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:47:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637245  77 58448 82071589 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:47:51 [DEBUG   ] Got command RPT from MCS: ref #637246
2018-11-26 22:47:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:47:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637246  12 58448 82071781 A  ERRORBžff'
2018-11-26 22:48:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 22:48:10 [DEBUG   ] Got command BAM from MCS: ref #637247
2018-11-26 22:48:10 [INFO    ] Finished processing BAM request
2018-11-26 22:48:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:48:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637247   8 58448 82090033 A  ERROR'
2018-11-26 22:48:10 [DEBUG   ] Got command BAM from MCS: ref #637248
2018-11-26 22:48:10 [INFO    ] Finished processing BAM request
2018-11-26 22:48:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:48:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637248   8 58448 82090037 A  ERROR'
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 22:48:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 22:48:10 [DEBUG   ] spawn_process: at 2018-11-26 22:48:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 22:48:26 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 22:48:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:48:41 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 22:48:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:48:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:48:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:48:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:48:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:48:41 [DEBUG   ] Finished updating board flags in 2.191 seconds
2018-11-26 22:48:46 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 22:48:51 [DEBUG   ] Got command BAM from MCS: ref #637249
2018-11-26 22:48:51 [INFO    ] Finished processing BAM request
2018-11-26 22:48:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:48:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637249   8 58448 82131025 A  ERROR'
2018-11-26 22:48:51 [DEBUG   ] Got command BAM from MCS: ref #637250
2018-11-26 22:48:51 [INFO    ] Finished processing BAM request
2018-11-26 22:48:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:48:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637250   8 58448 82131029 A  ERROR'
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 22:48:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 22:48:51 [DEBUG   ] spawn_process: at 2018-11-26 22:48:51 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 22:49:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 22:49:26 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:49:32 [DEBUG   ] Got command BAM from MCS: ref #637256
2018-11-26 22:49:32 [INFO    ] Finished processing BAM request
2018-11-26 22:49:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:49:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637256   8 58448 82172033 A  ERROR'
2018-11-26 22:49:32 [DEBUG   ] Got command BAM from MCS: ref #637255
2018-11-26 22:49:32 [INFO    ] Finished processing BAM request
2018-11-26 22:49:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:49:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637255   8 58448 82172037 A  ERROR'
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 22:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 22:49:32 [DEBUG   ] spawn_process: at 2018-11-26 22:49:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 22:49:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:49:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:49:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:49:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:49:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:49:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:49:41 [DEBUG   ] Finished updating board flags in 2.006 seconds
2018-11-26 22:49:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 22:50:06 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 22:50:12 [DEBUG   ] Got command BAM from MCS: ref #637258
2018-11-26 22:50:12 [INFO    ] Finished processing BAM request
2018-11-26 22:50:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:50:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637258   8 58448 82212025 A  ERROR'
2018-11-26 22:50:12 [DEBUG   ] Got command BAM from MCS: ref #637257
2018-11-26 22:50:12 [INFO    ] Finished processing BAM request
2018-11-26 22:50:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:50:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637257   8 58448 82212029 A  ERROR'
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 22:50:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 22:50:12 [DEBUG   ] spawn_process: at 2018-11-26 22:50:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 22:50:26 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 22:50:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:50:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:50:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:50:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:50:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:50:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:50:41 [DEBUG   ] Finished updating board flags in 1.969 seconds
2018-11-26 22:50:47 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 22:50:48 [DEBUG   ] Got command RPT from MCS: ref #637259
2018-11-26 22:50:48 [DEBUG   ] summary = ERROR
2018-11-26 22:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637259  13 58448 82248833 A  ERRORERROR'
2018-11-26 22:50:48 [DEBUG   ] Got command RPT from MCS: ref #637260
2018-11-26 22:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637260  77 58448 82248834 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:50:52 [DEBUG   ] Got command BAM from MCS: ref #637277
2018-11-26 22:50:52 [INFO    ] Finished processing BAM request
2018-11-26 22:50:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:50:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637277   8 58448 82252029 A  ERROR'
2018-11-26 22:50:52 [DEBUG   ] Got command BAM from MCS: ref #637276
2018-11-26 22:50:52 [INFO    ] Finished processing BAM request
2018-11-26 22:50:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:50:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637276   8 58448 82252033 A  ERROR'
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 22:50:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 22:50:52 [DEBUG   ] spawn_process: at 2018-11-26 22:50:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 22:51:07 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 22:51:27 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 22:51:32 [DEBUG   ] Got command BAM from MCS: ref #637279
2018-11-26 22:51:32 [INFO    ] Finished processing BAM request
2018-11-26 22:51:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:51:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637279   8 58448 82292030 A  ERROR'
2018-11-26 22:51:32 [DEBUG   ] Got command BAM from MCS: ref #637278
2018-11-26 22:51:32 [INFO    ] Finished processing BAM request
2018-11-26 22:51:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:51:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637278   8 58448 82292034 A  ERROR'
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 22:51:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 22:51:32 [DEBUG   ] spawn_process: at 2018-11-26 22:51:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 22:51:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:51:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:51:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:51:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:51:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:51:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:51:41 [DEBUG   ] Finished updating board flags in 1.817 seconds
2018-11-26 22:51:47 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 22:52:07 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 22:52:12 [DEBUG   ] Got command BAM from MCS: ref #637285
2018-11-26 22:52:12 [INFO    ] Finished processing BAM request
2018-11-26 22:52:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:52:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637285   8 58448 82332033 A  ERROR'
2018-11-26 22:52:12 [DEBUG   ] Got command BAM from MCS: ref #637284
2018-11-26 22:52:12 [INFO    ] Finished processing BAM request
2018-11-26 22:52:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:52:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637284   8 58448 82332037 A  ERROR'
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 22:52:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 22:52:12 [DEBUG   ] spawn_process: at 2018-11-26 22:52:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 22:52:27 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 22:52:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 22:52:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:52:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:52:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:52:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:52:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:52:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:52:41 [DEBUG   ] Finished updating board flags in 1.790 seconds
2018-11-26 22:52:47 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 22:52:51 [DEBUG   ] Got command RPT from MCS: ref #637290
2018-11-26 22:52:51 [DEBUG   ] summary = ERROR
2018-11-26 22:52:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637290  13 58448 82371069 A  ERRORERROR'
2018-11-26 22:52:51 [DEBUG   ] Got command RPT from MCS: ref #637291
2018-11-26 22:52:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:52:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637291  77 58448 82371273 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:52:51 [DEBUG   ] Got command RPT from MCS: ref #637292
2018-11-26 22:52:51 [DEBUG   ] summary = ERROR
2018-11-26 22:52:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637292  13 58448 82371465 A  ERRORERROR'
2018-11-26 22:52:51 [DEBUG   ] Got command RPT from MCS: ref #637294
2018-11-26 22:52:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:52:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637294  77 58448 82371669 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:52:51 [DEBUG   ] Got command RPT from MCS: ref #637295
2018-11-26 22:52:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:52:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637295  12 58448 82371873 A  ERRORBžff'
2018-11-26 22:52:52 [DEBUG   ] Got command BAM from MCS: ref #637289
2018-11-26 22:52:52 [INFO    ] Finished processing BAM request
2018-11-26 22:52:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:52:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637289   8 58448 82372033 A  ERROR'
2018-11-26 22:52:52 [DEBUG   ] Got command BAM from MCS: ref #637288
2018-11-26 22:52:52 [INFO    ] Finished processing BAM request
2018-11-26 22:52:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:52:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637288   8 58448 82372037 A  ERROR'
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 22:52:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 22:52:52 [DEBUG   ] spawn_process: at 2018-11-26 22:52:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 22:53:07 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 22:53:27 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 22:53:32 [DEBUG   ] Got command BAM from MCS: ref #637301
2018-11-26 22:53:32 [INFO    ] Finished processing BAM request
2018-11-26 22:53:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:53:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637301   8 58448 82412029 A  ERROR'
2018-11-26 22:53:32 [DEBUG   ] Got command BAM from MCS: ref #637300
2018-11-26 22:53:32 [INFO    ] Finished processing BAM request
2018-11-26 22:53:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:53:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637300   8 58448 82412033 A  ERROR'
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 22:53:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 22:53:32 [DEBUG   ] spawn_process: at 2018-11-26 22:53:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 22:53:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 22:53:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:53:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 22:53:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:53:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:53:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:53:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:53:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:53:41 [DEBUG   ] Finished updating board flags in 1.613 seconds
2018-11-26 22:53:47 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 22:54:07 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 22:54:12 [DEBUG   ] Got command BAM from MCS: ref #637303
2018-11-26 22:54:12 [INFO    ] Finished processing BAM request
2018-11-26 22:54:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:54:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637303   8 58448 82452029 A  ERROR'
2018-11-26 22:54:12 [DEBUG   ] Got command BAM from MCS: ref #637302
2018-11-26 22:54:12 [INFO    ] Finished processing BAM request
2018-11-26 22:54:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:54:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637302   8 58448 82452033 A  ERROR'
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 22:54:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 22:54:12 [DEBUG   ] spawn_process: at 2018-11-26 22:54:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 22:54:27 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 22:54:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:54:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-26 22:54:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:54:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:54:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:54:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:54:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:54:41 [DEBUG   ] Finished updating board flags in 1.693 seconds
2018-11-26 22:54:47 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 22:54:52 [DEBUG   ] Got command BAM from MCS: ref #637305
2018-11-26 22:54:52 [INFO    ] Finished processing BAM request
2018-11-26 22:54:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:54:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637305   8 58448 82492025 A  ERROR'
2018-11-26 22:54:52 [DEBUG   ] Got command BAM from MCS: ref #637304
2018-11-26 22:54:52 [INFO    ] Finished processing BAM request
2018-11-26 22:54:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:54:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637304   8 58448 82492029 A  ERROR'
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 22:54:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 22:54:52 [DEBUG   ] spawn_process: at 2018-11-26 22:54:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 22:55:07 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 22:55:27 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 22:55:32 [DEBUG   ] Got command BAM from MCS: ref #637310
2018-11-26 22:55:32 [INFO    ] Finished processing BAM request
2018-11-26 22:55:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:55:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637310   8 58448 82532029 A  ERROR'
2018-11-26 22:55:32 [DEBUG   ] Got command BAM from MCS: ref #637311
2018-11-26 22:55:32 [INFO    ] Finished processing BAM request
2018-11-26 22:55:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:55:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637311   8 58448 82532033 A  ERROR'
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 22:55:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 22:55:32 [DEBUG   ] spawn_process: at 2018-11-26 22:55:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 22:55:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:55:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-26 22:55:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:55:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:55:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:55:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:55:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:55:41 [DEBUG   ] Finished updating board flags in 1.490 seconds
2018-11-26 22:55:47 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 22:55:48 [DEBUG   ] Got command RPT from MCS: ref #637312
2018-11-26 22:55:48 [DEBUG   ] summary = ERROR
2018-11-26 22:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637312  13 58448 82548829 A  ERRORERROR'
2018-11-26 22:55:48 [DEBUG   ] Got command RPT from MCS: ref #637313
2018-11-26 22:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637313  77 58448 82548830 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:56:07 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-26 22:56:12 [DEBUG   ] Got command BAM from MCS: ref #637330
2018-11-26 22:56:12 [INFO    ] Finished processing BAM request
2018-11-26 22:56:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:56:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637330   8 58448 82572029 A  ERROR'
2018-11-26 22:56:12 [DEBUG   ] Got command BAM from MCS: ref #637329
2018-11-26 22:56:12 [INFO    ] Finished processing BAM request
2018-11-26 22:56:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:56:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637329   8 58448 82572033 A  ERROR'
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 22:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 22:56:12 [DEBUG   ] spawn_process: at 2018-11-26 22:56:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 22:56:27 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 22:56:40 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 22:56:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:56:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:56:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:56:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:56:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:56:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:56:41 [DEBUG   ] Finished updating board flags in 1.459 seconds
2018-11-26 22:56:47 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 22:56:52 [DEBUG   ] Got command BAM from MCS: ref #637331
2018-11-26 22:56:52 [INFO    ] Finished processing BAM request
2018-11-26 22:56:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:56:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637331   8 58448 82612033 A  ERROR'
2018-11-26 22:56:52 [DEBUG   ] Got command BAM from MCS: ref #637332
2018-11-26 22:56:52 [INFO    ] Finished processing BAM request
2018-11-26 22:56:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:56:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637332   8 58448 82612037 A  ERROR'
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 22:56:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 22:56:52 [DEBUG   ] spawn_process: at 2018-11-26 22:56:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 22:57:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 22:57:27 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 22:57:32 [DEBUG   ] Got command BAM from MCS: ref #637334
2018-11-26 22:57:32 [INFO    ] Finished processing BAM request
2018-11-26 22:57:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:57:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637334   8 58448 82652033 A  ERROR'
2018-11-26 22:57:32 [DEBUG   ] Got command BAM from MCS: ref #637333
2018-11-26 22:57:32 [INFO    ] Finished processing BAM request
2018-11-26 22:57:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:57:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637333   8 58448 82652037 A  ERROR'
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 22:57:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 22:57:32 [DEBUG   ] spawn_process: at 2018-11-26 22:57:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 22:57:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 22:57:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:57:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:57:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:57:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:57:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:57:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:57:41 [DEBUG   ] Finished updating board flags in 1.420 seconds
2018-11-26 22:57:47 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2018-11-26 22:57:51 [DEBUG   ] Got command RPT from MCS: ref #637335
2018-11-26 22:57:51 [DEBUG   ] summary = ERROR
2018-11-26 22:57:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637335  13 58448 82671153 A  ERRORERROR'
2018-11-26 22:57:51 [DEBUG   ] Got command RPT from MCS: ref #637336
2018-11-26 22:57:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:57:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637336  77 58448 82671357 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:57:51 [DEBUG   ] Got command RPT from MCS: ref #637337
2018-11-26 22:57:51 [DEBUG   ] summary = ERROR
2018-11-26 22:57:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637337  13 58448 82671561 A  ERRORERROR'
2018-11-26 22:57:51 [DEBUG   ] Got command RPT from MCS: ref #637338
2018-11-26 22:57:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 22:57:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637338  77 58448 82671753 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 22:57:51 [DEBUG   ] Got command RPT from MCS: ref #637339
2018-11-26 22:57:51 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 22:57:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637339  12 58448 82671957 A  ERRORBŸff'
2018-11-26 22:58:08 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-26 22:58:12 [DEBUG   ] Got command BAM from MCS: ref #637345
2018-11-26 22:58:12 [INFO    ] Finished processing BAM request
2018-11-26 22:58:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:58:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637345   8 58448 82692025 A  ERROR'
2018-11-26 22:58:12 [DEBUG   ] Got command BAM from MCS: ref #637344
2018-11-26 22:58:12 [INFO    ] Finished processing BAM request
2018-11-26 22:58:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:58:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637344   8 58448 82692029 A  ERROR'
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 22:58:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 22:58:12 [DEBUG   ] spawn_process: at 2018-11-26 22:58:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 22:58:28 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2018-11-26 22:58:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:58:41 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 22:58:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 1
2018-11-26 22:58:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:58:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:58:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:58:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:58:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:58:41 [DEBUG   ] Finished updating board flags in 1.422 seconds
2018-11-26 22:58:48 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 22:58:52 [DEBUG   ] Got command BAM from MCS: ref #637347
2018-11-26 22:58:52 [INFO    ] Finished processing BAM request
2018-11-26 22:58:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:58:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637347   8 58448 82732025 A  ERROR'
2018-11-26 22:58:52 [DEBUG   ] Got command BAM from MCS: ref #637346
2018-11-26 22:58:52 [INFO    ] Finished processing BAM request
2018-11-26 22:58:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:58:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637346   8 58448 82732029 A  ERROR'
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 22:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 22:58:52 [DEBUG   ] spawn_process: at 2018-11-26 22:58:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 22:59:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 22:59:28 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 22:59:32 [DEBUG   ] Got command BAM from MCS: ref #637348
2018-11-26 22:59:32 [INFO    ] Finished processing BAM request
2018-11-26 22:59:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:59:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637348   8 58448 82772025 A  ERROR'
2018-11-26 22:59:32 [DEBUG   ] Got command BAM from MCS: ref #637349
2018-11-26 22:59:32 [INFO    ] Finished processing BAM request
2018-11-26 22:59:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 22:59:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637349   8 58448 82772029 A  ERROR'
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 22:59:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 22:59:32 [DEBUG   ] spawn_process: at 2018-11-26 22:59:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 22:59:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 22:59:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:59:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:59:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:59:41 [WARNING ] (13, 1, 'bot')
2018-11-26 22:59:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 22:59:41 [DEBUG   ] Finished updating board flags in 1.424 seconds
2018-11-26 22:59:48 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 23:00:08 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 23:00:12 [DEBUG   ] Got command BAM from MCS: ref #637351
2018-11-26 23:00:12 [INFO    ] Finished processing BAM request
2018-11-26 23:00:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:00:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637351   8 58448 82812025 A  ERROR'
2018-11-26 23:00:12 [DEBUG   ] Got command BAM from MCS: ref #637350
2018-11-26 23:00:12 [INFO    ] Finished processing BAM request
2018-11-26 23:00:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:00:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637350   8 58448 82812029 A  ERROR'
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:00:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:00:12 [DEBUG   ] spawn_process: at 2018-11-26 23:00:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:00:28 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 23:00:40 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:00:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:00:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-26 23:00:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:00:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:00:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:00:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:00:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:00:41 [DEBUG   ] Finished updating board flags in 1.431 seconds
2018-11-26 23:00:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 23:00:48 [DEBUG   ] Got command RPT from MCS: ref #637352
2018-11-26 23:00:48 [DEBUG   ] summary = ERROR
2018-11-26 23:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637352  13 58448 82848833 A  ERRORERROR'
2018-11-26 23:00:48 [DEBUG   ] Got command RPT from MCS: ref #637353
2018-11-26 23:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637353  77 58448 82848834 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:00:52 [DEBUG   ] Got command BAM from MCS: ref #637369
2018-11-26 23:00:52 [INFO    ] Finished processing BAM request
2018-11-26 23:00:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:00:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637369   8 58448 82852029 A  ERROR'
2018-11-26 23:00:52 [DEBUG   ] Got command BAM from MCS: ref #637370
2018-11-26 23:00:52 [INFO    ] Finished processing BAM request
2018-11-26 23:00:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:00:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637370   8 58448 82852033 A  ERROR'
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:00:52 [DEBUG   ] spawn_process: at 2018-11-26 23:00:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:01:08 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-26 23:01:28 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 23:01:32 [DEBUG   ] Got command BAM from MCS: ref #637375
2018-11-26 23:01:32 [INFO    ] Finished processing BAM request
2018-11-26 23:01:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:01:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637375   8 58448 82892033 A  ERROR'
2018-11-26 23:01:32 [DEBUG   ] Got command BAM from MCS: ref #637376
2018-11-26 23:01:32 [INFO    ] Finished processing BAM request
2018-11-26 23:01:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:01:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637376   8 58448 82892037 A  ERROR'
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:01:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:01:32 [DEBUG   ] spawn_process: at 2018-11-26 23:01:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:01:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:01:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:01:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:01:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:01:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:01:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:01:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:01:41 [DEBUG   ] Finished updating board flags in 1.424 seconds
2018-11-26 23:01:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-26 23:02:08 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 23:02:12 [DEBUG   ] Got command BAM from MCS: ref #637378
2018-11-26 23:02:12 [INFO    ] Finished processing BAM request
2018-11-26 23:02:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:02:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637378   8 58448 82932025 A  ERROR'
2018-11-26 23:02:12 [DEBUG   ] Got command BAM from MCS: ref #637377
2018-11-26 23:02:12 [INFO    ] Finished processing BAM request
2018-11-26 23:02:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:02:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637377   8 58448 82932029 A  ERROR'
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:02:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:02:12 [DEBUG   ] spawn_process: at 2018-11-26 23:02:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:02:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 23:02:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-26 23:02:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:02:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:02:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:02:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:02:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:02:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:02:41 [DEBUG   ] Finished updating board flags in 1.430 seconds
2018-11-26 23:02:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 23:02:51 [DEBUG   ] Got command RPT from MCS: ref #637383
2018-11-26 23:02:51 [DEBUG   ] summary = ERROR
2018-11-26 23:02:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637383  13 58448 82971241 A  ERRORERROR'
2018-11-26 23:02:51 [DEBUG   ] Got command RPT from MCS: ref #637384
2018-11-26 23:02:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:02:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637384  77 58448 82971445 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:02:51 [DEBUG   ] Got command RPT from MCS: ref #637385
2018-11-26 23:02:51 [DEBUG   ] summary = ERROR
2018-11-26 23:02:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637385  13 58448 82971637 A  ERRORERROR'
2018-11-26 23:02:51 [DEBUG   ] Got command RPT from MCS: ref #637387
2018-11-26 23:02:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:02:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637387  77 58448 82971841 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:02:52 [DEBUG   ] Got command BAM from MCS: ref #637382
2018-11-26 23:02:52 [INFO    ] Finished processing BAM request
2018-11-26 23:02:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:02:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637382   8 58448 82972025 A  ERROR'
2018-11-26 23:02:52 [DEBUG   ] Got command BAM from MCS: ref #637381
2018-11-26 23:02:52 [INFO    ] Finished processing BAM request
2018-11-26 23:02:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:02:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637381   8 58448 82972029 A  ERROR'
2018-11-26 23:02:52 [DEBUG   ] Got command RPT from MCS: ref #637388
2018-11-26 23:02:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:02:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637388  12 58448 82972041 A  ERRORBžff'
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:02:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:02:52 [DEBUG   ] spawn_process: at 2018-11-26 23:02:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:03:08 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 23:03:29 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-26 23:03:32 [DEBUG   ] Got command BAM from MCS: ref #637393
2018-11-26 23:03:32 [INFO    ] Finished processing BAM request
2018-11-26 23:03:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:03:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637393   8 58448 83012033 A  ERROR'
2018-11-26 23:03:32 [DEBUG   ] Got command BAM from MCS: ref #637394
2018-11-26 23:03:32 [INFO    ] Finished processing BAM request
2018-11-26 23:03:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:03:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637394   8 58448 83012037 A  ERROR'
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:03:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:03:32 [DEBUG   ] spawn_process: at 2018-11-26 23:03:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:03:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:03:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:03:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:03:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:03:41 [WARNING ] (13, 1, 'bot')
2018-11-26 23:03:41 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:03:41 [DEBUG   ] Finished updating board flags in 1.429 seconds
2018-11-26 23:03:48 [DEBUG   ] Finished updating ant stats in 0.405 seconds
2018-11-26 23:04:09 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2018-11-26 23:04:12 [DEBUG   ] Got command BAM from MCS: ref #637399
2018-11-26 23:04:12 [INFO    ] Finished processing BAM request
2018-11-26 23:04:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:04:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637399   8 58448 83052033 A  ERROR'
2018-11-26 23:04:12 [DEBUG   ] Got command BAM from MCS: ref #637400
2018-11-26 23:04:12 [INFO    ] Finished processing BAM request
2018-11-26 23:04:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:04:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637400   8 58448 83052037 A  ERROR'
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:04:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:04:12 [DEBUG   ] spawn_process: at 2018-11-26 23:04:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:04:29 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 23:04:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:04:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 23:04:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:04:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:04:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:04:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:04:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:04:42 [DEBUG   ] Finished updating board flags in 2.243 seconds
2018-11-26 23:04:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 23:04:52 [DEBUG   ] Got command BAM from MCS: ref #637402
2018-11-26 23:04:52 [INFO    ] Finished processing BAM request
2018-11-26 23:04:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:04:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637402   8 58448 83092033 A  ERROR'
2018-11-26 23:04:52 [DEBUG   ] Got command BAM from MCS: ref #637401
2018-11-26 23:04:52 [INFO    ] Finished processing BAM request
2018-11-26 23:04:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:04:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637401   8 58448 83092037 A  ERROR'
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:04:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:04:52 [DEBUG   ] spawn_process: at 2018-11-26 23:04:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:05:09 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 23:05:29 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 23:05:32 [DEBUG   ] Got command BAM from MCS: ref #637403
2018-11-26 23:05:32 [INFO    ] Finished processing BAM request
2018-11-26 23:05:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:05:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637403   8 58448 83132033 A  ERROR'
2018-11-26 23:05:32 [DEBUG   ] Got command BAM from MCS: ref #637404
2018-11-26 23:05:32 [INFO    ] Finished processing BAM request
2018-11-26 23:05:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:05:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637404   8 58448 83132037 A  ERROR'
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:05:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:05:32 [DEBUG   ] spawn_process: at 2018-11-26 23:05:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:05:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:05:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:05:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:05:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:05:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:05:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:05:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:05:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:05:42 [DEBUG   ] Finished updating board flags in 2.210 seconds
2018-11-26 23:05:48 [DEBUG   ] Got command RPT from MCS: ref #637405
2018-11-26 23:05:48 [DEBUG   ] summary = ERROR
2018-11-26 23:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637405  13 58448 83148829 A  ERRORERROR'
2018-11-26 23:05:48 [DEBUG   ] Got command RPT from MCS: ref #637406
2018-11-26 23:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637406  77 58448 83148830 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:05:49 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 23:06:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 23:06:12 [DEBUG   ] Got command BAM from MCS: ref #637422
2018-11-26 23:06:12 [INFO    ] Finished processing BAM request
2018-11-26 23:06:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:06:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637422   8 58448 83172029 A  ERROR'
2018-11-26 23:06:12 [DEBUG   ] Got command BAM from MCS: ref #637423
2018-11-26 23:06:12 [INFO    ] Finished processing BAM request
2018-11-26 23:06:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:06:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637423   8 58448 83172033 A  ERROR'
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:06:12 [DEBUG   ] spawn_process: at 2018-11-26 23:06:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:06:29 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-26 23:06:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:06:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:06:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:06:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:06:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:06:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:06:42 [DEBUG   ] Finished updating board flags in 2.159 seconds
2018-11-26 23:06:49 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 23:06:52 [DEBUG   ] Got command BAM from MCS: ref #637425
2018-11-26 23:06:52 [INFO    ] Finished processing BAM request
2018-11-26 23:06:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:06:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637425   8 58448 83212025 A  ERROR'
2018-11-26 23:06:52 [DEBUG   ] Got command BAM from MCS: ref #637424
2018-11-26 23:06:52 [INFO    ] Finished processing BAM request
2018-11-26 23:06:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:06:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637424   8 58448 83212029 A  ERROR'
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:06:52 [DEBUG   ] spawn_process: at 2018-11-26 23:06:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:07:09 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 23:07:29 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 23:07:31 [DEBUG   ] Got command BAM from MCS: ref #637430
2018-11-26 23:07:31 [INFO    ] Finished processing BAM request
2018-11-26 23:07:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:07:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637430   8 58448 83251029 A  ERROR'
2018-11-26 23:07:31 [DEBUG   ] Got command BAM from MCS: ref #637431
2018-11-26 23:07:31 [INFO    ] Finished processing BAM request
2018-11-26 23:07:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:07:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637431   8 58448 83251033 A  ERROR'
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:07:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:07:31 [DEBUG   ] spawn_process: at 2018-11-26 23:07:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:07:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:07:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:07:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:07:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:07:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:07:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:07:42 [DEBUG   ] Finished updating board flags in 2.015 seconds
2018-11-26 23:07:49 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 23:07:51 [DEBUG   ] Got command RPT from MCS: ref #637432
2018-11-26 23:07:51 [DEBUG   ] summary = ERROR
2018-11-26 23:07:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637432  13 58448 83271325 A  ERRORERROR'
2018-11-26 23:07:51 [DEBUG   ] Got command RPT from MCS: ref #637433
2018-11-26 23:07:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:07:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637433  77 58448 83271533 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:07:51 [DEBUG   ] Got command RPT from MCS: ref #637434
2018-11-26 23:07:51 [DEBUG   ] summary = ERROR
2018-11-26 23:07:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637434  13 58448 83271737 A  ERRORERROR'
2018-11-26 23:07:51 [DEBUG   ] Got command RPT from MCS: ref #637435
2018-11-26 23:07:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:07:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637435  77 58448 83271929 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:07:52 [DEBUG   ] Got command RPT from MCS: ref #637436
2018-11-26 23:07:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:07:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637436  12 58448 83272133 A  ERRORBŸff'
2018-11-26 23:08:09 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 23:08:10 [DEBUG   ] Got command BAM from MCS: ref #637437
2018-11-26 23:08:10 [INFO    ] Finished processing BAM request
2018-11-26 23:08:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:08:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637437   8 58448 83290029 A  ERROR'
2018-11-26 23:08:10 [DEBUG   ] Got command BAM from MCS: ref #637438
2018-11-26 23:08:10 [INFO    ] Finished processing BAM request
2018-11-26 23:08:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:08:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637438   8 58448 83290033 A  ERROR'
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:08:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:08:10 [DEBUG   ] spawn_process: at 2018-11-26 23:08:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:08:29 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 23:08:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:08:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:08:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:08:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:08:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:08:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:08:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:08:42 [DEBUG   ] Finished updating board flags in 1.972 seconds
2018-11-26 23:08:49 [DEBUG   ] Got command BAM from MCS: ref #637439
2018-11-26 23:08:49 [INFO    ] Finished processing BAM request
2018-11-26 23:08:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:08:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637439   8 58448 83329025 A  ERROR'
2018-11-26 23:08:49 [DEBUG   ] Got command BAM from MCS: ref #637440
2018-11-26 23:08:49 [INFO    ] Finished processing BAM request
2018-11-26 23:08:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:08:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637440   8 58448 83329029 A  ERROR'
2018-11-26 23:08:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:08:49 [DEBUG   ] spawn_process: at 2018-11-26 23:08:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:09:09 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 23:09:28 [DEBUG   ] Got command BAM from MCS: ref #637441
2018-11-26 23:09:28 [INFO    ] Finished processing BAM request
2018-11-26 23:09:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:09:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637441   8 58448 83368033 A  ERROR'
2018-11-26 23:09:28 [DEBUG   ] Got command BAM from MCS: ref #637442
2018-11-26 23:09:28 [INFO    ] Finished processing BAM request
2018-11-26 23:09:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:09:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637442   8 58448 83368037 A  ERROR'
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:09:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:09:28 [DEBUG   ] spawn_process: at 2018-11-26 23:09:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:09:30 [DEBUG   ] Finished updating ant stats in 1.405 seconds
2018-11-26 23:09:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:09:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 23:09:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:09:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:09:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:09:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:09:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:09:42 [DEBUG   ] Finished updating board flags in 1.810 seconds
2018-11-26 23:09:50 [DEBUG   ] Finished updating ant stats in 1.392 seconds
2018-11-26 23:10:07 [DEBUG   ] Got command BAM from MCS: ref #637447
2018-11-26 23:10:07 [INFO    ] Finished processing BAM request
2018-11-26 23:10:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:10:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637447   8 58448 83407025 A  ERROR'
2018-11-26 23:10:07 [DEBUG   ] Got command BAM from MCS: ref #637448
2018-11-26 23:10:07 [INFO    ] Finished processing BAM request
2018-11-26 23:10:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:10:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637448   8 58448 83407029 A  ERROR'
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:10:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:10:07 [DEBUG   ] spawn_process: at 2018-11-26 23:10:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:10:10 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-26 23:10:30 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 23:10:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:10:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:10:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:10:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:10:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:10:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:10:42 [DEBUG   ] Finished updating board flags in 1.813 seconds
2018-11-26 23:10:46 [DEBUG   ] Got command BAM from MCS: ref #637450
2018-11-26 23:10:46 [INFO    ] Finished processing BAM request
2018-11-26 23:10:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:10:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637450   8 58448 83446029 A  ERROR'
2018-11-26 23:10:46 [DEBUG   ] Got command BAM from MCS: ref #637449
2018-11-26 23:10:46 [INFO    ] Finished processing BAM request
2018-11-26 23:10:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:10:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637449   8 58448 83446033 A  ERROR'
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:10:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:10:46 [DEBUG   ] spawn_process: at 2018-11-26 23:10:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:10:48 [DEBUG   ] Got command RPT from MCS: ref #637451
2018-11-26 23:10:48 [DEBUG   ] summary = ERROR
2018-11-26 23:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637451  13 58448 83448833 A  ERRORERROR'
2018-11-26 23:10:48 [DEBUG   ] Got command RPT from MCS: ref #637452
2018-11-26 23:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637452  77 58448 83448834 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:10:50 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 23:11:10 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 23:11:25 [DEBUG   ] Got command BAM from MCS: ref #637469
2018-11-26 23:11:25 [INFO    ] Finished processing BAM request
2018-11-26 23:11:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:11:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637469   8 58448 83485033 A  ERROR'
2018-11-26 23:11:25 [DEBUG   ] Got command BAM from MCS: ref #637468
2018-11-26 23:11:25 [INFO    ] Finished processing BAM request
2018-11-26 23:11:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:11:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637468   8 58448 83485037 A  ERROR'
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:11:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:11:25 [DEBUG   ] spawn_process: at 2018-11-26 23:11:25 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:11:30 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-26 23:11:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:11:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:11:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:11:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:11:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:11:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:11:42 [DEBUG   ] Finished updating board flags in 1.795 seconds
2018-11-26 23:11:50 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 23:12:04 [DEBUG   ] Got command BAM from MCS: ref #637470
2018-11-26 23:12:04 [INFO    ] Finished processing BAM request
2018-11-26 23:12:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:12:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637470   8 58448 83524029 A  ERROR'
2018-11-26 23:12:04 [DEBUG   ] Got command BAM from MCS: ref #637471
2018-11-26 23:12:04 [INFO    ] Finished processing BAM request
2018-11-26 23:12:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:12:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637471   8 58448 83524033 A  ERROR'
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:12:04 [DEBUG   ] spawn_process: at 2018-11-26 23:12:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:12:10 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 23:12:30 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 23:12:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:12:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:12:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:12:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:12:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:12:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:12:42 [DEBUG   ] Finished updating board flags in 1.610 seconds
2018-11-26 23:12:43 [DEBUG   ] Got command BAM from MCS: ref #637475
2018-11-26 23:12:43 [INFO    ] Finished processing BAM request
2018-11-26 23:12:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:12:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637475   8 58448 83563029 A  ERROR'
2018-11-26 23:12:43 [DEBUG   ] Got command BAM from MCS: ref #637474
2018-11-26 23:12:43 [INFO    ] Finished processing BAM request
2018-11-26 23:12:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:12:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637474   8 58448 83563033 A  ERROR'
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:12:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:12:43 [DEBUG   ] spawn_process: at 2018-11-26 23:12:43 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:12:50 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 23:12:51 [DEBUG   ] Got command RPT from MCS: ref #637476
2018-11-26 23:12:51 [DEBUG   ] summary = ERROR
2018-11-26 23:12:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637476  13 58448 83571413 A  ERRORERROR'
2018-11-26 23:12:51 [DEBUG   ] Got command RPT from MCS: ref #637477
2018-11-26 23:12:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:12:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637477  77 58448 83571617 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:12:51 [DEBUG   ] Got command RPT from MCS: ref #637478
2018-11-26 23:12:51 [DEBUG   ] summary = ERROR
2018-11-26 23:12:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637478  13 58448 83571821 A  ERRORERROR'
2018-11-26 23:12:52 [DEBUG   ] Got command RPT from MCS: ref #637480
2018-11-26 23:12:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:12:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637480  77 58448 83572013 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:12:52 [DEBUG   ] Got command RPT from MCS: ref #637482
2018-11-26 23:12:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:12:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637482  12 58448 83572217 A  ERRORBŸff'
2018-11-26 23:13:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-26 23:13:22 [DEBUG   ] Got command BAM from MCS: ref #637491
2018-11-26 23:13:22 [INFO    ] Finished processing BAM request
2018-11-26 23:13:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:13:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637491   8 58448 83602029 A  ERROR'
2018-11-26 23:13:22 [DEBUG   ] Got command BAM from MCS: ref #637490
2018-11-26 23:13:22 [INFO    ] Finished processing BAM request
2018-11-26 23:13:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:13:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637490   8 58448 83602033 A  ERROR'
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:13:22 [DEBUG   ] spawn_process: at 2018-11-26 23:13:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:13:30 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-26 23:13:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:13:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:13:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:13:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:13:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:13:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:13:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:13:42 [DEBUG   ] Finished updating board flags in 1.592 seconds
2018-11-26 23:13:50 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 23:14:01 [DEBUG   ] Got command BAM from MCS: ref #637492
2018-11-26 23:14:01 [INFO    ] Finished processing BAM request
2018-11-26 23:14:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:14:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637492   8 58448 83641025 A  ERROR'
2018-11-26 23:14:01 [DEBUG   ] Got command BAM from MCS: ref #637493
2018-11-26 23:14:01 [INFO    ] Finished processing BAM request
2018-11-26 23:14:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:14:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637493   8 58448 83641029 A  ERROR'
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:14:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:14:01 [DEBUG   ] spawn_process: at 2018-11-26 23:14:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:14:10 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 23:14:30 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 23:14:40 [DEBUG   ] Got command BAM from MCS: ref #637494
2018-11-26 23:14:40 [INFO    ] Finished processing BAM request
2018-11-26 23:14:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:14:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637494   8 58448 83680033 A  ERROR'
2018-11-26 23:14:40 [DEBUG   ] Got command BAM from MCS: ref #637495
2018-11-26 23:14:40 [INFO    ] Finished processing BAM request
2018-11-26 23:14:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:14:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637495   8 58448 83680037 A  ERROR'
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:14:40 [DEBUG   ] spawn_process: at 2018-11-26 23:14:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:14:41 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:14:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:14:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:14:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:14:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:14:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:14:42 [DEBUG   ] Finished updating board flags in 1.394 seconds
2018-11-26 23:14:50 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 23:15:10 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 23:15:19 [DEBUG   ] Got command BAM from MCS: ref #637496
2018-11-26 23:15:19 [INFO    ] Finished processing BAM request
2018-11-26 23:15:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:15:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637496   8 58448 83719025 A  ERROR'
2018-11-26 23:15:19 [DEBUG   ] Got command BAM from MCS: ref #637497
2018-11-26 23:15:19 [INFO    ] Finished processing BAM request
2018-11-26 23:15:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:15:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637497   8 58448 83719029 A  ERROR'
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:15:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:15:19 [DEBUG   ] spawn_process: at 2018-11-26 23:15:19 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:15:30 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 23:15:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:15:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:15:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:15:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:15:42 [WARNING ] (13, 1, 'bot')
2018-11-26 23:15:42 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:15:42 [DEBUG   ] Finished updating board flags in 1.372 seconds
2018-11-26 23:15:48 [DEBUG   ] Got command RPT from MCS: ref #637498
2018-11-26 23:15:48 [DEBUG   ] summary = ERROR
2018-11-26 23:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637498  13 58448 83748841 A  ERRORERROR'
2018-11-26 23:15:48 [DEBUG   ] Got command RPT from MCS: ref #637499
2018-11-26 23:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637499  77 58448 83748842 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:15:51 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 23:15:58 [DEBUG   ] Got command BAM from MCS: ref #637516
2018-11-26 23:15:58 [INFO    ] Finished processing BAM request
2018-11-26 23:15:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637516   8 58448 83758025 A  ERROR'
2018-11-26 23:15:58 [DEBUG   ] Got command BAM from MCS: ref #637515
2018-11-26 23:15:58 [INFO    ] Finished processing BAM request
2018-11-26 23:15:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:15:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637515   8 58448 83758029 A  ERROR'
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:15:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:15:58 [DEBUG   ] spawn_process: at 2018-11-26 23:15:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:16:11 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 23:16:31 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 23:16:37 [DEBUG   ] Got command BAM from MCS: ref #637521
2018-11-26 23:16:37 [INFO    ] Finished processing BAM request
2018-11-26 23:16:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:16:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637521   8 58448 83797025 A  ERROR'
2018-11-26 23:16:37 [DEBUG   ] Got command BAM from MCS: ref #637522
2018-11-26 23:16:37 [INFO    ] Finished processing BAM request
2018-11-26 23:16:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:16:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637522   8 58448 83797029 A  ERROR'
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:16:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:16:37 [DEBUG   ] spawn_process: at 2018-11-26 23:16:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:16:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:16:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-26 23:16:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:16:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 23:16:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-26 23:16:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:16:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:16:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:16:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:16:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:16:43 [DEBUG   ] Finished updating board flags in 2.038 seconds
2018-11-26 23:16:51 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 23:17:11 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 23:17:16 [DEBUG   ] Got command BAM from MCS: ref #637524
2018-11-26 23:17:16 [INFO    ] Finished processing BAM request
2018-11-26 23:17:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:17:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637524   8 58448 83836029 A  ERROR'
2018-11-26 23:17:16 [DEBUG   ] Got command BAM from MCS: ref #637523
2018-11-26 23:17:16 [INFO    ] Finished processing BAM request
2018-11-26 23:17:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:17:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637523   8 58448 83836033 A  ERROR'
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:17:16 [DEBUG   ] spawn_process: at 2018-11-26 23:17:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:17:31 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2018-11-26 23:17:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:17:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-26 23:17:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:17:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:17:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:17:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:17:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:17:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:17:43 [DEBUG   ] Finished updating board flags in 2.011 seconds
2018-11-26 23:17:51 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-26 23:17:51 [DEBUG   ] Got command RPT from MCS: ref #637525
2018-11-26 23:17:51 [DEBUG   ] summary = ERROR
2018-11-26 23:17:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637525  13 58448 83871505 A  ERRORERROR'
2018-11-26 23:17:51 [DEBUG   ] Got command RPT from MCS: ref #637526
2018-11-26 23:17:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:17:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637526  77 58448 83871697 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:17:51 [DEBUG   ] Got command RPT from MCS: ref #637527
2018-11-26 23:17:51 [DEBUG   ] summary = ERROR
2018-11-26 23:17:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637527  13 58448 83871901 A  ERRORERROR'
2018-11-26 23:17:52 [DEBUG   ] Got command RPT from MCS: ref #637528
2018-11-26 23:17:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:17:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637528  77 58448 83872105 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:17:52 [DEBUG   ] Got command RPT from MCS: ref #637529
2018-11-26 23:17:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:17:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637529  12 58448 83872309 A  ERRORBžff'
2018-11-26 23:17:55 [DEBUG   ] Got command BAM from MCS: ref #637530
2018-11-26 23:17:55 [INFO    ] Finished processing BAM request
2018-11-26 23:17:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:17:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637530   8 58448 83875025 A  ERROR'
2018-11-26 23:17:55 [DEBUG   ] Got command BAM from MCS: ref #637531
2018-11-26 23:17:55 [INFO    ] Finished processing BAM request
2018-11-26 23:17:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:17:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637531   8 58448 83875029 A  ERROR'
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:17:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:17:55 [DEBUG   ] spawn_process: at 2018-11-26 23:17:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:18:11 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 23:18:31 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 23:18:34 [DEBUG   ] Got command BAM from MCS: ref #637533
2018-11-26 23:18:34 [INFO    ] Finished processing BAM request
2018-11-26 23:18:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:18:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637533   8 58448 83914033 A  ERROR'
2018-11-26 23:18:34 [DEBUG   ] Got command BAM from MCS: ref #637532
2018-11-26 23:18:34 [INFO    ] Finished processing BAM request
2018-11-26 23:18:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:18:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637532   8 58448 83914037 A  ERROR'
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:18:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:18:34 [DEBUG   ] spawn_process: at 2018-11-26 23:18:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:18:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:18:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:18:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:18:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:18:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:18:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:18:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:18:43 [DEBUG   ] Finished updating board flags in 1.983 seconds
2018-11-26 23:18:51 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2018-11-26 23:19:12 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-26 23:19:13 [DEBUG   ] Got command BAM from MCS: ref #637538
2018-11-26 23:19:13 [INFO    ] Finished processing BAM request
2018-11-26 23:19:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:19:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637538   8 58448 83953033 A  ERROR'
2018-11-26 23:19:13 [DEBUG   ] Got command BAM from MCS: ref #637539
2018-11-26 23:19:13 [INFO    ] Finished processing BAM request
2018-11-26 23:19:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:19:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637539   8 58448 83953037 A  ERROR'
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:19:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:19:13 [DEBUG   ] spawn_process: at 2018-11-26 23:19:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:19:32 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 23:19:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:19:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:19:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:19:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:19:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:19:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:19:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:19:43 [DEBUG   ] Finished updating board flags in 1.813 seconds
2018-11-26 23:19:52 [DEBUG   ] Got command BAM from MCS: ref #637541
2018-11-26 23:19:52 [INFO    ] Finished processing BAM request
2018-11-26 23:19:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:19:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637541   8 58448 83992025 A  ERROR'
2018-11-26 23:19:52 [DEBUG   ] Got command BAM from MCS: ref #637540
2018-11-26 23:19:52 [INFO    ] Finished processing BAM request
2018-11-26 23:19:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:19:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637540   8 58448 83992029 A  ERROR'
2018-11-26 23:19:52 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:19:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:19:52 [DEBUG   ] spawn_process: at 2018-11-26 23:19:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:20:12 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 23:20:31 [DEBUG   ] Got command BAM from MCS: ref #637542
2018-11-26 23:20:31 [INFO    ] Finished processing BAM request
2018-11-26 23:20:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:20:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637542   8 58448 84031033 A  ERROR'
2018-11-26 23:20:31 [DEBUG   ] Got command BAM from MCS: ref #637543
2018-11-26 23:20:31 [INFO    ] Finished processing BAM request
2018-11-26 23:20:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:20:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637543   8 58448 84031037 A  ERROR'
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:20:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:20:31 [DEBUG   ] spawn_process: at 2018-11-26 23:20:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:20:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 23:20:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:20:43 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 23:20:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:20:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:20:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:20:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:20:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:20:43 [DEBUG   ] Finished updating board flags in 1.805 seconds
2018-11-26 23:20:48 [DEBUG   ] Got command RPT from MCS: ref #637544
2018-11-26 23:20:48 [DEBUG   ] summary = ERROR
2018-11-26 23:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637544  13 58448 84048841 A  ERRORERROR'
2018-11-26 23:20:48 [DEBUG   ] Got command RPT from MCS: ref #637545
2018-11-26 23:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637545  77 58448 84048842 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:20:52 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 23:21:10 [DEBUG   ] Got command BAM from MCS: ref #637561
2018-11-26 23:21:10 [INFO    ] Finished processing BAM request
2018-11-26 23:21:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:21:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637561   8 58448 84070033 A  ERROR'
2018-11-26 23:21:10 [DEBUG   ] Got command BAM from MCS: ref #637562
2018-11-26 23:21:10 [INFO    ] Finished processing BAM request
2018-11-26 23:21:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:21:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637562   8 58448 84070037 A  ERROR'
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:21:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:21:10 [DEBUG   ] spawn_process: at 2018-11-26 23:21:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:21:12 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 23:21:33 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2018-11-26 23:21:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:21:43 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-26 23:21:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:21:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:21:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:21:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:21:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:21:43 [DEBUG   ] Finished updating board flags in 1.791 seconds
2018-11-26 23:21:49 [DEBUG   ] Got command BAM from MCS: ref #637564
2018-11-26 23:21:49 [INFO    ] Finished processing BAM request
2018-11-26 23:21:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:21:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637564   8 58448 84109033 A  ERROR'
2018-11-26 23:21:49 [DEBUG   ] Got command BAM from MCS: ref #637563
2018-11-26 23:21:49 [INFO    ] Finished processing BAM request
2018-11-26 23:21:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:21:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637563   8 58448 84109037 A  ERROR'
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:21:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:21:49 [DEBUG   ] spawn_process: at 2018-11-26 23:21:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:21:53 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-26 23:22:13 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 23:22:28 [DEBUG   ] Got command BAM from MCS: ref #637569
2018-11-26 23:22:28 [INFO    ] Finished processing BAM request
2018-11-26 23:22:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:22:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637569   8 58448 84148033 A  ERROR'
2018-11-26 23:22:28 [DEBUG   ] Got command BAM from MCS: ref #637570
2018-11-26 23:22:28 [INFO    ] Finished processing BAM request
2018-11-26 23:22:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:22:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637570   8 58448 84148037 A  ERROR'
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:22:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:22:28 [DEBUG   ] spawn_process: at 2018-11-26 23:22:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:22:33 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 23:22:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:22:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:22:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-26 23:22:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:22:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:22:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:22:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:22:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:22:43 [DEBUG   ] Finished updating board flags in 1.619 seconds
2018-11-26 23:22:51 [DEBUG   ] Got command RPT from MCS: ref #637573
2018-11-26 23:22:51 [DEBUG   ] summary = ERROR
2018-11-26 23:22:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637573  13 58448 84171585 A  ERRORERROR'
2018-11-26 23:22:51 [DEBUG   ] Got command RPT from MCS: ref #637574
2018-11-26 23:22:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:22:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637574  77 58448 84171789 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:22:51 [DEBUG   ] Got command RPT from MCS: ref #637575
2018-11-26 23:22:51 [DEBUG   ] summary = ERROR
2018-11-26 23:22:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637575  13 58448 84171993 A  ERRORERROR'
2018-11-26 23:22:52 [DEBUG   ] Got command RPT from MCS: ref #637577
2018-11-26 23:22:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:22:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637577  77 58448 84172197 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:22:52 [DEBUG   ] Got command RPT from MCS: ref #637578
2018-11-26 23:22:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:22:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637578  12 58448 84172390 A  ERRORBŸff'
2018-11-26 23:22:53 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 23:23:07 [DEBUG   ] Got command BAM from MCS: ref #637584
2018-11-26 23:23:07 [INFO    ] Finished processing BAM request
2018-11-26 23:23:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:23:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637584   8 58448 84187033 A  ERROR'
2018-11-26 23:23:07 [DEBUG   ] Got command BAM from MCS: ref #637583
2018-11-26 23:23:07 [INFO    ] Finished processing BAM request
2018-11-26 23:23:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:23:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637583   8 58448 84187037 A  ERROR'
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:23:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:23:07 [DEBUG   ] spawn_process: at 2018-11-26 23:23:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:23:13 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 23:23:33 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 23:23:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:23:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:23:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:23:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:23:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:23:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:23:43 [DEBUG   ] Finished updating board flags in 1.587 seconds
2018-11-26 23:23:46 [DEBUG   ] Got command BAM from MCS: ref #637586
2018-11-26 23:23:46 [INFO    ] Finished processing BAM request
2018-11-26 23:23:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:23:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637586   8 58448 84226033 A  ERROR'
2018-11-26 23:23:46 [DEBUG   ] Got command BAM from MCS: ref #637585
2018-11-26 23:23:46 [INFO    ] Finished processing BAM request
2018-11-26 23:23:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:23:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637585   8 58448 84226037 A  ERROR'
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:23:46 [DEBUG   ] spawn_process: at 2018-11-26 23:23:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:23:53 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-26 23:24:13 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-26 23:24:25 [DEBUG   ] Got command BAM from MCS: ref #637588
2018-11-26 23:24:25 [INFO    ] Finished processing BAM request
2018-11-26 23:24:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:24:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637588   8 58448 84265025 A  ERROR'
2018-11-26 23:24:25 [DEBUG   ] Got command BAM from MCS: ref #637587
2018-11-26 23:24:25 [INFO    ] Finished processing BAM request
2018-11-26 23:24:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:24:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637587   8 58448 84265029 A  ERROR'
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:24:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:24:25 [DEBUG   ] spawn_process: at 2018-11-26 23:24:25 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:24:33 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 23:24:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:24:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:24:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:24:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:24:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:24:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:24:43 [DEBUG   ] Finished updating board flags in 1.409 seconds
2018-11-26 23:24:53 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 23:25:04 [DEBUG   ] Got command BAM from MCS: ref #637593
2018-11-26 23:25:04 [INFO    ] Finished processing BAM request
2018-11-26 23:25:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:25:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637593   8 58448 84304029 A  ERROR'
2018-11-26 23:25:04 [DEBUG   ] Got command BAM from MCS: ref #637594
2018-11-26 23:25:04 [INFO    ] Finished processing BAM request
2018-11-26 23:25:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:25:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637594   8 58448 84304033 A  ERROR'
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:25:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:25:04 [DEBUG   ] spawn_process: at 2018-11-26 23:25:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:25:13 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 23:25:33 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 23:25:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:25:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:25:43 [DEBUG   ] Got command BAM from MCS: ref #637595
2018-11-26 23:25:43 [INFO    ] Finished processing BAM request
2018-11-26 23:25:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:25:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637595   8 58448 84343033 A  ERROR'
2018-11-26 23:25:43 [DEBUG   ] Got command BAM from MCS: ref #637596
2018-11-26 23:25:43 [INFO    ] Finished processing BAM request
2018-11-26 23:25:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:25:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637596   8 58448 84343041 A  ERROR'
2018-11-26 23:25:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:25:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:25:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:25:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:25:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:25:43 [DEBUG   ] Finished updating board flags in 1.411 seconds
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:25:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:25:43 [DEBUG   ] spawn_process: at 2018-11-26 23:25:43 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:25:48 [DEBUG   ] Got command RPT from MCS: ref #637597
2018-11-26 23:25:48 [DEBUG   ] summary = ERROR
2018-11-26 23:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637597  13 58448 84348833 A  ERRORERROR'
2018-11-26 23:25:48 [DEBUG   ] Got command RPT from MCS: ref #637598
2018-11-26 23:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637598  77 58448 84348834 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:25:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 23:26:13 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-26 23:26:22 [DEBUG   ] Got command BAM from MCS: ref #637615
2018-11-26 23:26:22 [INFO    ] Finished processing BAM request
2018-11-26 23:26:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:26:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637615   8 58448 84382033 A  ERROR'
2018-11-26 23:26:22 [DEBUG   ] Got command BAM from MCS: ref #637614
2018-11-26 23:26:22 [INFO    ] Finished processing BAM request
2018-11-26 23:26:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:26:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637614   8 58448 84382037 A  ERROR'
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:26:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:26:22 [DEBUG   ] spawn_process: at 2018-11-26 23:26:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:26:33 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 23:26:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:26:42 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:26:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:26:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:26:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:26:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:26:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:26:43 [DEBUG   ] Finished updating board flags in 1.382 seconds
2018-11-26 23:26:54 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2018-11-26 23:27:01 [DEBUG   ] Got command BAM from MCS: ref #637617
2018-11-26 23:27:01 [INFO    ] Finished processing BAM request
2018-11-26 23:27:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:27:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637617   8 58448 84421033 A  ERROR'
2018-11-26 23:27:01 [DEBUG   ] Got command BAM from MCS: ref #637616
2018-11-26 23:27:01 [INFO    ] Finished processing BAM request
2018-11-26 23:27:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:27:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637616   8 58448 84421037 A  ERROR'
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:27:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:27:01 [DEBUG   ] spawn_process: at 2018-11-26 23:27:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:27:14 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-26 23:27:34 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 23:27:40 [DEBUG   ] Got command BAM from MCS: ref #637618
2018-11-26 23:27:40 [INFO    ] Finished processing BAM request
2018-11-26 23:27:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:27:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637618   8 58448 84460034 A  ERROR'
2018-11-26 23:27:40 [DEBUG   ] Got command BAM from MCS: ref #637619
2018-11-26 23:27:40 [INFO    ] Finished processing BAM request
2018-11-26 23:27:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:27:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637619   8 58448 84460038 A  ERROR'
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:27:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:27:40 [DEBUG   ] spawn_process: at 2018-11-26 23:27:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:27:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:27:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:27:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:27:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:27:43 [WARNING ] (13, 1, 'bot')
2018-11-26 23:27:43 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:27:43 [DEBUG   ] Finished updating board flags in 1.341 seconds
2018-11-26 23:27:51 [DEBUG   ] Got command RPT from MCS: ref #637620
2018-11-26 23:27:51 [DEBUG   ] summary = ERROR
2018-11-26 23:27:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637620  13 58448 84471677 A  ERRORERROR'
2018-11-26 23:27:51 [DEBUG   ] Got command RPT from MCS: ref #637621
2018-11-26 23:27:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:27:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637621  77 58448 84471881 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:27:52 [DEBUG   ] Got command RPT from MCS: ref #637622
2018-11-26 23:27:52 [DEBUG   ] summary = ERROR
2018-11-26 23:27:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637622  13 58448 84472074 A  ERRORERROR'
2018-11-26 23:27:52 [DEBUG   ] Got command RPT from MCS: ref #637623
2018-11-26 23:27:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:27:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637623  77 58448 84472277 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:27:52 [DEBUG   ] Got command RPT from MCS: ref #637624
2018-11-26 23:27:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:27:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637624  12 58448 84472481 A  ERRORBŸff'
2018-11-26 23:27:54 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 23:28:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 23:28:19 [DEBUG   ] Got command BAM from MCS: ref #637629
2018-11-26 23:28:19 [INFO    ] Finished processing BAM request
2018-11-26 23:28:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:28:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637629   8 58448 84499030 A  ERROR'
2018-11-26 23:28:19 [DEBUG   ] Got command BAM from MCS: ref #637630
2018-11-26 23:28:19 [INFO    ] Finished processing BAM request
2018-11-26 23:28:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:28:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637630   8 58448 84499034 A  ERROR'
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:28:19 [DEBUG   ] spawn_process: at 2018-11-26 23:28:19 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:28:34 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-26 23:28:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:28:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:28:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:28:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:28:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:28:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:28:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:28:44 [DEBUG   ] Finished updating board flags in 2.094 seconds
2018-11-26 23:28:54 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 23:28:58 [DEBUG   ] Got command BAM from MCS: ref #637632
2018-11-26 23:28:58 [INFO    ] Finished processing BAM request
2018-11-26 23:28:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:28:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637632   8 58448 84538026 A  ERROR'
2018-11-26 23:28:58 [DEBUG   ] Got command BAM from MCS: ref #637631
2018-11-26 23:28:58 [INFO    ] Finished processing BAM request
2018-11-26 23:28:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:28:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637631   8 58448 84538030 A  ERROR'
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:28:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:28:58 [DEBUG   ] spawn_process: at 2018-11-26 23:28:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:29:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 23:29:34 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 23:29:37 [DEBUG   ] Got command BAM from MCS: ref #637634
2018-11-26 23:29:37 [INFO    ] Finished processing BAM request
2018-11-26 23:29:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:29:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637634   8 58448 84577025 A  ERROR'
2018-11-26 23:29:37 [DEBUG   ] Got command BAM from MCS: ref #637633
2018-11-26 23:29:37 [INFO    ] Finished processing BAM request
2018-11-26 23:29:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:29:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637633   8 58448 84577029 A  ERROR'
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:29:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:29:37 [DEBUG   ] spawn_process: at 2018-11-26 23:29:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:29:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:29:44 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 23:29:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:29:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:29:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:29:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:29:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:29:44 [DEBUG   ] Finished updating board flags in 1.984 seconds
2018-11-26 23:29:54 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 23:30:14 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 23:30:16 [DEBUG   ] Got command BAM from MCS: ref #637636
2018-11-26 23:30:16 [INFO    ] Finished processing BAM request
2018-11-26 23:30:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:30:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637636   8 58448 84616029 A  ERROR'
2018-11-26 23:30:16 [DEBUG   ] Got command BAM from MCS: ref #637635
2018-11-26 23:30:16 [INFO    ] Finished processing BAM request
2018-11-26 23:30:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:30:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637635   8 58448 84616033 A  ERROR'
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:30:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:30:16 [DEBUG   ] spawn_process: at 2018-11-26 23:30:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:30:34 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 23:30:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:30:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:30:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:30:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:30:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:30:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:30:44 [DEBUG   ] Finished updating board flags in 1.990 seconds
2018-11-26 23:30:48 [DEBUG   ] Got command RPT from MCS: ref #637637
2018-11-26 23:30:48 [DEBUG   ] summary = ERROR
2018-11-26 23:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637637  13 58448 84648833 A  ERRORERROR'
2018-11-26 23:30:48 [DEBUG   ] Got command RPT from MCS: ref #637638
2018-11-26 23:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637638  77 58448 84648834 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:30:54 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 23:30:55 [DEBUG   ] Got command BAM from MCS: ref #637654
2018-11-26 23:30:55 [INFO    ] Finished processing BAM request
2018-11-26 23:30:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:30:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637654   8 58448 84655033 A  ERROR'
2018-11-26 23:30:55 [DEBUG   ] Got command BAM from MCS: ref #637655
2018-11-26 23:30:55 [INFO    ] Finished processing BAM request
2018-11-26 23:30:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:30:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637655   8 58448 84655037 A  ERROR'
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:30:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:30:55 [DEBUG   ] spawn_process: at 2018-11-26 23:30:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:31:15 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2018-11-26 23:31:34 [DEBUG   ] Got command BAM from MCS: ref #637661
2018-11-26 23:31:34 [INFO    ] Finished processing BAM request
2018-11-26 23:31:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:31:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637661   8 58448 84694033 A  ERROR'
2018-11-26 23:31:34 [DEBUG   ] Got command BAM from MCS: ref #637660
2018-11-26 23:31:34 [INFO    ] Finished processing BAM request
2018-11-26 23:31:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:31:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637660   8 58448 84694037 A  ERROR'
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:31:34 [DEBUG   ] spawn_process: at 2018-11-26 23:31:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:31:35 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2018-11-26 23:31:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:31:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:31:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:31:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:31:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:31:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:31:44 [DEBUG   ] Finished updating board flags in 1.633 seconds
2018-11-26 23:31:55 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 23:32:13 [DEBUG   ] Got command BAM from MCS: ref #637662
2018-11-26 23:32:13 [INFO    ] Finished processing BAM request
2018-11-26 23:32:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:32:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637662   8 58448 84733033 A  ERROR'
2018-11-26 23:32:13 [DEBUG   ] Got command BAM from MCS: ref #637663
2018-11-26 23:32:13 [INFO    ] Finished processing BAM request
2018-11-26 23:32:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:32:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637663   8 58448 84733037 A  ERROR'
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:32:13 [DEBUG   ] spawn_process: at 2018-11-26 23:32:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:32:15 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 23:32:35 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 23:32:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:32:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:32:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:32:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:32:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:32:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:32:44 [DEBUG   ] Finished updating board flags in 1.701 seconds
2018-11-26 23:32:51 [DEBUG   ] Got command RPT from MCS: ref #637668
2018-11-26 23:32:51 [DEBUG   ] summary = ERROR
2018-11-26 23:32:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637668  13 58448 84771765 A  ERRORERROR'
2018-11-26 23:32:51 [DEBUG   ] Got command RPT from MCS: ref #637669
2018-11-26 23:32:51 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:32:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637669  77 58448 84771969 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:32:52 [DEBUG   ] Got command BAM from MCS: ref #637667
2018-11-26 23:32:52 [INFO    ] Finished processing BAM request
2018-11-26 23:32:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:32:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637667   8 58448 84772033 A  ERROR'
2018-11-26 23:32:52 [DEBUG   ] Got command BAM from MCS: ref #637666
2018-11-26 23:32:52 [INFO    ] Finished processing BAM request
2018-11-26 23:32:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:32:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637666   8 58448 84772037 A  ERROR'
2018-11-26 23:32:52 [DEBUG   ] Got command RPT from MCS: ref #637670
2018-11-26 23:32:52 [DEBUG   ] summary = ERROR
2018-11-26 23:32:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637670  13 58448 84772169 A  ERRORERROR'
2018-11-26 23:32:52 [DEBUG   ] Got command RPT from MCS: ref #637672
2018-11-26 23:32:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:32:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637672  77 58448 84772361 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:32:52 [DEBUG   ] Got command RPT from MCS: ref #637674
2018-11-26 23:32:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:32:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637674  12 58448 84772565 A  ERRORBŸff'
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:32:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:32:52 [DEBUG   ] spawn_process: at 2018-11-26 23:32:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:32:55 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 23:33:15 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 23:33:31 [DEBUG   ] Got command BAM from MCS: ref #637678
2018-11-26 23:33:31 [INFO    ] Finished processing BAM request
2018-11-26 23:33:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:33:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637678   8 58448 84811033 A  ERROR'
2018-11-26 23:33:31 [DEBUG   ] Got command BAM from MCS: ref #637679
2018-11-26 23:33:31 [INFO    ] Finished processing BAM request
2018-11-26 23:33:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:33:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637679   8 58448 84811037 A  ERROR'
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:33:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:33:31 [DEBUG   ] spawn_process: at 2018-11-26 23:33:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:33:35 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 23:33:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:33:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:33:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:33:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:33:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:33:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:33:44 [DEBUG   ] Finished updating board flags in 1.479 seconds
2018-11-26 23:33:56 [DEBUG   ] Finished updating ant stats in 1.605 seconds
2018-11-26 23:34:10 [DEBUG   ] Got command BAM from MCS: ref #637685
2018-11-26 23:34:10 [INFO    ] Finished processing BAM request
2018-11-26 23:34:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:34:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637685   8 58448 84850029 A  ERROR'
2018-11-26 23:34:10 [DEBUG   ] Got command BAM from MCS: ref #637684
2018-11-26 23:34:10 [INFO    ] Finished processing BAM request
2018-11-26 23:34:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:34:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637684   8 58448 84850037 A  ERROR'
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:34:10 [DEBUG   ] spawn_process: at 2018-11-26 23:34:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:34:15 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 23:34:36 [DEBUG   ] Finished updating ant stats in 1.599 seconds
2018-11-26 23:34:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:34:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:34:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:34:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:34:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:34:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:34:44 [DEBUG   ] Finished updating board flags in 1.401 seconds
2018-11-26 23:34:49 [DEBUG   ] Got command BAM from MCS: ref #637686
2018-11-26 23:34:49 [INFO    ] Finished processing BAM request
2018-11-26 23:34:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:34:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637686   8 58448 84889029 A  ERROR'
2018-11-26 23:34:49 [DEBUG   ] Got command BAM from MCS: ref #637687
2018-11-26 23:34:49 [INFO    ] Finished processing BAM request
2018-11-26 23:34:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:34:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637687   8 58448 84889033 A  ERROR'
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:34:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:34:49 [DEBUG   ] spawn_process: at 2018-11-26 23:34:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:34:55 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 23:35:16 [DEBUG   ] Finished updating ant stats in 1.405 seconds
2018-11-26 23:35:28 [DEBUG   ] Got command BAM from MCS: ref #637689
2018-11-26 23:35:28 [INFO    ] Finished processing BAM request
2018-11-26 23:35:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:35:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637689   8 58448 84928033 A  ERROR'
2018-11-26 23:35:28 [DEBUG   ] Got command BAM from MCS: ref #637688
2018-11-26 23:35:28 [INFO    ] Finished processing BAM request
2018-11-26 23:35:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:35:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637688   8 58448 84928037 A  ERROR'
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:35:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:35:28 [DEBUG   ] spawn_process: at 2018-11-26 23:35:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:35:35 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-26 23:35:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:35:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:35:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:35:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:35:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:35:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:35:44 [DEBUG   ] Finished updating board flags in 1.447 seconds
2018-11-26 23:35:48 [DEBUG   ] Got command RPT from MCS: ref #637690
2018-11-26 23:35:48 [DEBUG   ] summary = ERROR
2018-11-26 23:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637690  13 58448 84948845 A  ERRORERROR'
2018-11-26 23:35:48 [DEBUG   ] Got command RPT from MCS: ref #637691
2018-11-26 23:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637691  77 58448 84948846 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:35:56 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-26 23:36:07 [DEBUG   ] Got command BAM from MCS: ref #637707
2018-11-26 23:36:07 [INFO    ] Finished processing BAM request
2018-11-26 23:36:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:36:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637707   8 58448 84967029 A  ERROR'
2018-11-26 23:36:07 [DEBUG   ] Got command BAM from MCS: ref #637708
2018-11-26 23:36:07 [INFO    ] Finished processing BAM request
2018-11-26 23:36:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:36:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637708   8 58448 84967033 A  ERROR'
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:36:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:36:07 [DEBUG   ] spawn_process: at 2018-11-26 23:36:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:36:16 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 23:36:36 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 23:36:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:36:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:36:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:36:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:36:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:36:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:36:44 [DEBUG   ] Finished updating board flags in 1.443 seconds
2018-11-26 23:36:46 [DEBUG   ] Got command BAM from MCS: ref #637709
2018-11-26 23:36:46 [INFO    ] Finished processing BAM request
2018-11-26 23:36:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:36:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637709   8 58448 85006033 A  ERROR'
2018-11-26 23:36:46 [DEBUG   ] Got command BAM from MCS: ref #637710
2018-11-26 23:36:46 [INFO    ] Finished processing BAM request
2018-11-26 23:36:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:36:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637710   8 58448 85006037 A  ERROR'
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:36:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:36:46 [DEBUG   ] spawn_process: at 2018-11-26 23:36:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:36:56 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2018-11-26 23:37:16 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 23:37:25 [DEBUG   ] Got command BAM from MCS: ref #637715
2018-11-26 23:37:25 [INFO    ] Finished processing BAM request
2018-11-26 23:37:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:37:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637715   8 58448 85045025 A  ERROR'
2018-11-26 23:37:25 [DEBUG   ] Got command BAM from MCS: ref #637716
2018-11-26 23:37:25 [INFO    ] Finished processing BAM request
2018-11-26 23:37:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:37:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637716   8 58448 85045029 A  ERROR'
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:37:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:37:25 [DEBUG   ] spawn_process: at 2018-11-26 23:37:25 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:37:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 23:37:43 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:37:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:37:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:37:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:37:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:37:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:37:44 [DEBUG   ] Finished updating board flags in 1.445 seconds
2018-11-26 23:37:51 [DEBUG   ] Got command RPT from MCS: ref #637717
2018-11-26 23:37:51 [DEBUG   ] summary = ERROR
2018-11-26 23:37:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637717  13 58448 85071849 A  ERRORERROR'
2018-11-26 23:37:52 [DEBUG   ] Got command RPT from MCS: ref #637718
2018-11-26 23:37:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:37:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637718  77 58448 85072053 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:37:52 [DEBUG   ] Got command RPT from MCS: ref #637719
2018-11-26 23:37:52 [DEBUG   ] summary = ERROR
2018-11-26 23:37:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637719  13 58448 85072257 A  ERRORERROR'
2018-11-26 23:37:52 [DEBUG   ] Got command RPT from MCS: ref #637720
2018-11-26 23:37:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:37:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637720  77 58448 85072449 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:37:52 [DEBUG   ] Got command RPT from MCS: ref #637721
2018-11-26 23:37:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:37:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637721  12 58448 85072653 A  ERRORBžff'
2018-11-26 23:37:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 23:38:04 [DEBUG   ] Got command BAM from MCS: ref #637722
2018-11-26 23:38:04 [INFO    ] Finished processing BAM request
2018-11-26 23:38:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:38:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637722   8 58448 85084029 A  ERROR'
2018-11-26 23:38:04 [DEBUG   ] Got command BAM from MCS: ref #637723
2018-11-26 23:38:04 [INFO    ] Finished processing BAM request
2018-11-26 23:38:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:38:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637723   8 58448 85084033 A  ERROR'
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:38:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:38:04 [DEBUG   ] spawn_process: at 2018-11-26 23:38:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:38:16 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 23:38:36 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 23:38:43 [DEBUG   ] Got command BAM from MCS: ref #637725
2018-11-26 23:38:43 [INFO    ] Finished processing BAM request
2018-11-26 23:38:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:38:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637725   8 58448 85123034 A  ERROR'
2018-11-26 23:38:43 [DEBUG   ] Got command BAM from MCS: ref #637724
2018-11-26 23:38:43 [INFO    ] Finished processing BAM request
2018-11-26 23:38:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:38:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637724   8 58448 85123038 A  ERROR'
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:38:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:38:43 [DEBUG   ] spawn_process: at 2018-11-26 23:38:43 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:38:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:38:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:38:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:38:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:38:44 [WARNING ] (13, 1, 'bot')
2018-11-26 23:38:44 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:38:44 [DEBUG   ] Finished updating board flags in 1.484 seconds
2018-11-26 23:38:56 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 23:39:16 [DEBUG   ] Finished updating ant stats in 0.405 seconds
2018-11-26 23:39:22 [DEBUG   ] Got command BAM from MCS: ref #637726
2018-11-26 23:39:22 [INFO    ] Finished processing BAM request
2018-11-26 23:39:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:39:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637726   8 58448 85162029 A  ERROR'
2018-11-26 23:39:22 [DEBUG   ] Got command BAM from MCS: ref #637727
2018-11-26 23:39:22 [INFO    ] Finished processing BAM request
2018-11-26 23:39:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:39:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637727   8 58448 85162033 A  ERROR'
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:39:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:39:22 [DEBUG   ] spawn_process: at 2018-11-26 23:39:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:39:37 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2018-11-26 23:39:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:39:45 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 23:39:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:39:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:39:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:39:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:39:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:39:45 [DEBUG   ] Finished updating board flags in 2.262 seconds
2018-11-26 23:39:57 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 23:40:01 [DEBUG   ] Got command BAM from MCS: ref #637729
2018-11-26 23:40:01 [INFO    ] Finished processing BAM request
2018-11-26 23:40:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:40:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637729   8 58448 85201025 A  ERROR'
2018-11-26 23:40:01 [DEBUG   ] Got command BAM from MCS: ref #637728
2018-11-26 23:40:01 [INFO    ] Finished processing BAM request
2018-11-26 23:40:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:40:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637728   8 58448 85201029 A  ERROR'
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:40:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:40:01 [DEBUG   ] spawn_process: at 2018-11-26 23:40:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:40:17 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-26 23:40:37 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 23:40:40 [DEBUG   ] Got command BAM from MCS: ref #637735
2018-11-26 23:40:40 [INFO    ] Finished processing BAM request
2018-11-26 23:40:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:40:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637735   8 58448 85240033 A  ERROR'
2018-11-26 23:40:40 [DEBUG   ] Got command BAM from MCS: ref #637734
2018-11-26 23:40:40 [INFO    ] Finished processing BAM request
2018-11-26 23:40:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:40:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637734   8 58448 85240037 A  ERROR'
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:40:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:40:40 [DEBUG   ] spawn_process: at 2018-11-26 23:40:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:40:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:40:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:40:45 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 23:40:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:40:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:40:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:40:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:40:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:40:45 [DEBUG   ] Finished updating board flags in 2.133 seconds
2018-11-26 23:40:48 [DEBUG   ] Got command RPT from MCS: ref #637736
2018-11-26 23:40:48 [DEBUG   ] summary = ERROR
2018-11-26 23:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637736  13 58448 85248837 A  ERRORERROR'
2018-11-26 23:40:48 [DEBUG   ] Got command RPT from MCS: ref #637737
2018-11-26 23:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637737  77 58448 85248838 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:40:57 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 23:41:17 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 23:41:19 [DEBUG   ] Got command BAM from MCS: ref #637754
2018-11-26 23:41:19 [INFO    ] Finished processing BAM request
2018-11-26 23:41:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:41:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637754   8 58448 85279033 A  ERROR'
2018-11-26 23:41:19 [DEBUG   ] Got command BAM from MCS: ref #637753
2018-11-26 23:41:19 [INFO    ] Finished processing BAM request
2018-11-26 23:41:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:41:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637753   8 58448 85279037 A  ERROR'
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:41:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:41:19 [DEBUG   ] spawn_process: at 2018-11-26 23:41:19 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:41:37 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-26 23:41:44 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:41:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:41:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:41:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:41:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:41:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:41:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:41:45 [DEBUG   ] Finished updating board flags in 2.006 seconds
2018-11-26 23:41:57 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-26 23:41:58 [DEBUG   ] Got command BAM from MCS: ref #637756
2018-11-26 23:41:58 [INFO    ] Finished processing BAM request
2018-11-26 23:41:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637756   8 58448 85318025 A  ERROR'
2018-11-26 23:41:58 [DEBUG   ] Got command BAM from MCS: ref #637755
2018-11-26 23:41:58 [INFO    ] Finished processing BAM request
2018-11-26 23:41:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:41:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637755   8 58448 85318029 A  ERROR'
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:41:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:41:58 [DEBUG   ] spawn_process: at 2018-11-26 23:41:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:42:17 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-26 23:42:37 [DEBUG   ] Got command BAM from MCS: ref #637760
2018-11-26 23:42:37 [INFO    ] Finished processing BAM request
2018-11-26 23:42:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:42:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637760   8 58448 85357029 A  ERROR'
2018-11-26 23:42:37 [DEBUG   ] Got command BAM from MCS: ref #637759
2018-11-26 23:42:37 [INFO    ] Finished processing BAM request
2018-11-26 23:42:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:42:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637759   8 58448 85357033 A  ERROR'
2018-11-26 23:42:37 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:42:37 [DEBUG   ] spawn_process: at 2018-11-26 23:42:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:42:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:42:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:42:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:42:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:42:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:42:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:42:45 [DEBUG   ] Finished updating board flags in 1.985 seconds
2018-11-26 23:42:51 [DEBUG   ] Got command RPT from MCS: ref #637761
2018-11-26 23:42:51 [DEBUG   ] summary = ERROR
2018-11-26 23:42:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637761  13 58448 85371937 A  ERRORERROR'
2018-11-26 23:42:52 [DEBUG   ] Got command RPT from MCS: ref #637762
2018-11-26 23:42:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:42:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637762  77 58448 85372141 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:42:52 [DEBUG   ] Got command RPT from MCS: ref #637763
2018-11-26 23:42:52 [DEBUG   ] summary = ERROR
2018-11-26 23:42:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637763  13 58448 85372345 A  ERRORERROR'
2018-11-26 23:42:52 [DEBUG   ] Got command RPT from MCS: ref #637765
2018-11-26 23:42:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:42:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637765  77 58448 85372537 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:42:52 [DEBUG   ] Got command RPT from MCS: ref #637767
2018-11-26 23:42:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:42:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637767  12 58448 85372741 A  ERRORBŸff'
2018-11-26 23:42:57 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 23:43:16 [DEBUG   ] Got command BAM from MCS: ref #637775
2018-11-26 23:43:16 [INFO    ] Finished processing BAM request
2018-11-26 23:43:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:43:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637775   8 58448 85396033 A  ERROR'
2018-11-26 23:43:16 [DEBUG   ] Got command BAM from MCS: ref #637776
2018-11-26 23:43:16 [INFO    ] Finished processing BAM request
2018-11-26 23:43:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:43:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637776   8 58448 85396037 A  ERROR'
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:43:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:43:16 [DEBUG   ] spawn_process: at 2018-11-26 23:43:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:43:17 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 23:43:37 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 23:43:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:43:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:43:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:43:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:43:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:43:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:43:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:43:45 [DEBUG   ] Finished updating board flags in 1.826 seconds
2018-11-26 23:43:55 [DEBUG   ] Got command BAM from MCS: ref #637777
2018-11-26 23:43:55 [INFO    ] Finished processing BAM request
2018-11-26 23:43:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:43:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637777   8 58448 85435033 A  ERROR'
2018-11-26 23:43:55 [DEBUG   ] Got command BAM from MCS: ref #637778
2018-11-26 23:43:55 [INFO    ] Finished processing BAM request
2018-11-26 23:43:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:43:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637778   8 58448 85435037 A  ERROR'
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:43:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:43:55 [DEBUG   ] spawn_process: at 2018-11-26 23:43:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:43:57 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 23:44:17 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 23:44:34 [DEBUG   ] Got command BAM from MCS: ref #637779
2018-11-26 23:44:34 [INFO    ] Finished processing BAM request
2018-11-26 23:44:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:44:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637779   8 58448 85474033 A  ERROR'
2018-11-26 23:44:34 [DEBUG   ] Got command BAM from MCS: ref #637780
2018-11-26 23:44:34 [INFO    ] Finished processing BAM request
2018-11-26 23:44:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:44:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637780   8 58448 85474037 A  ERROR'
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:44:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:44:34 [DEBUG   ] spawn_process: at 2018-11-26 23:44:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:44:37 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 23:44:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:44:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:44:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:44:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:44:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:44:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:44:45 [DEBUG   ] Finished updating board flags in 1.816 seconds
2018-11-26 23:44:57 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 23:45:13 [DEBUG   ] Got command BAM from MCS: ref #637782
2018-11-26 23:45:13 [INFO    ] Finished processing BAM request
2018-11-26 23:45:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:45:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637782   8 58448 85513025 A  ERROR'
2018-11-26 23:45:13 [DEBUG   ] Got command BAM from MCS: ref #637781
2018-11-26 23:45:13 [INFO    ] Finished processing BAM request
2018-11-26 23:45:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:45:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637781   8 58448 85513029 A  ERROR'
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:45:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:45:13 [DEBUG   ] spawn_process: at 2018-11-26 23:45:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:45:17 [DEBUG   ] Finished updating ant stats in 0.405 seconds
2018-11-26 23:45:38 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2018-11-26 23:45:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:45:45 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-26 23:45:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:45:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:45:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:45:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:45:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:45:45 [DEBUG   ] Finished updating board flags in 1.821 seconds
2018-11-26 23:45:48 [DEBUG   ] Got command RPT from MCS: ref #637783
2018-11-26 23:45:48 [DEBUG   ] summary = ERROR
2018-11-26 23:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637783  13 58448 85548841 A  ERRORERROR'
2018-11-26 23:45:48 [DEBUG   ] Got command RPT from MCS: ref #637784
2018-11-26 23:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637784  77 58448 85548842 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:45:52 [DEBUG   ] Got command BAM from MCS: ref #637801
2018-11-26 23:45:52 [INFO    ] Finished processing BAM request
2018-11-26 23:45:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:45:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637801   8 58448 85552025 A  ERROR'
2018-11-26 23:45:52 [DEBUG   ] Got command BAM from MCS: ref #637800
2018-11-26 23:45:52 [INFO    ] Finished processing BAM request
2018-11-26 23:45:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:45:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637800   8 58448 85552029 A  ERROR'
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:45:52 [DEBUG   ] spawn_process: at 2018-11-26 23:45:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:45:58 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 23:46:18 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 23:46:31 [DEBUG   ] Got command BAM from MCS: ref #637806
2018-11-26 23:46:31 [INFO    ] Finished processing BAM request
2018-11-26 23:46:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:46:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637806   8 58448 85591025 A  ERROR'
2018-11-26 23:46:31 [DEBUG   ] Got command BAM from MCS: ref #637807
2018-11-26 23:46:31 [INFO    ] Finished processing BAM request
2018-11-26 23:46:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:46:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637807   8 58448 85591029 A  ERROR'
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:46:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:46:31 [DEBUG   ] spawn_process: at 2018-11-26 23:46:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:46:38 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2018-11-26 23:46:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:46:45 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 23:46:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:46:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:46:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:46:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:46:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:46:45 [DEBUG   ] Finished updating board flags in 1.747 seconds
2018-11-26 23:46:58 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 23:47:10 [DEBUG   ] Got command BAM from MCS: ref #637808
2018-11-26 23:47:10 [INFO    ] Finished processing BAM request
2018-11-26 23:47:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:47:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637808   8 58448 85630025 A  ERROR'
2018-11-26 23:47:10 [DEBUG   ] Got command BAM from MCS: ref #637809
2018-11-26 23:47:10 [INFO    ] Finished processing BAM request
2018-11-26 23:47:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:47:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637809   8 58448 85630029 A  ERROR'
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:47:10 [DEBUG   ] spawn_process: at 2018-11-26 23:47:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:47:18 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-26 23:47:38 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 23:47:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:47:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 5
2018-11-26 23:47:45 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-26 23:47:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:47:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:47:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:47:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:47:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:47:45 [DEBUG   ] Finished updating board flags in 1.628 seconds
2018-11-26 23:47:49 [DEBUG   ] Got command BAM from MCS: ref #637810
2018-11-26 23:47:49 [INFO    ] Finished processing BAM request
2018-11-26 23:47:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637810   8 58448 85669033 A  ERROR'
2018-11-26 23:47:49 [DEBUG   ] Got command BAM from MCS: ref #637811
2018-11-26 23:47:49 [INFO    ] Finished processing BAM request
2018-11-26 23:47:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:47:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637811   8 58448 85669037 A  ERROR'
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:47:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:47:49 [DEBUG   ] spawn_process: at 2018-11-26 23:47:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:47:52 [DEBUG   ] Got command RPT from MCS: ref #637812
2018-11-26 23:47:52 [DEBUG   ] summary = ERROR
2018-11-26 23:47:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637812  13 58448 85672025 A  ERRORERROR'
2018-11-26 23:47:52 [DEBUG   ] Got command RPT from MCS: ref #637813
2018-11-26 23:47:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:47:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637813  77 58448 85672229 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:47:52 [DEBUG   ] Got command RPT from MCS: ref #637814
2018-11-26 23:47:52 [DEBUG   ] summary = ERROR
2018-11-26 23:47:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637814  13 58448 85672433 A  ERRORERROR'
2018-11-26 23:47:52 [DEBUG   ] Got command RPT from MCS: ref #637815
2018-11-26 23:47:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:47:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637815  77 58448 85672625 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:47:52 [DEBUG   ] Got command RPT from MCS: ref #637816
2018-11-26 23:47:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:47:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637816  12 58448 85672829 A  ERRORBžff'
2018-11-26 23:47:58 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 23:48:18 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 23:48:28 [DEBUG   ] Got command BAM from MCS: ref #637818
2018-11-26 23:48:28 [INFO    ] Finished processing BAM request
2018-11-26 23:48:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:48:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637818   8 58448 85708033 A  ERROR'
2018-11-26 23:48:28 [DEBUG   ] Got command BAM from MCS: ref #637817
2018-11-26 23:48:28 [INFO    ] Finished processing BAM request
2018-11-26 23:48:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:48:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637817   8 58448 85708037 A  ERROR'
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:48:28 [DEBUG   ] spawn_process: at 2018-11-26 23:48:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:48:38 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 23:48:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-26 23:48:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:48:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:48:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:48:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:48:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:48:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:48:45 [DEBUG   ] Finished updating board flags in 1.584 seconds
2018-11-26 23:48:58 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 23:49:07 [DEBUG   ] Got command BAM from MCS: ref #637824
2018-11-26 23:49:07 [INFO    ] Finished processing BAM request
2018-11-26 23:49:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:49:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637824   8 58448 85747029 A  ERROR'
2018-11-26 23:49:07 [DEBUG   ] Got command BAM from MCS: ref #637823
2018-11-26 23:49:07 [INFO    ] Finished processing BAM request
2018-11-26 23:49:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:49:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637823   8 58448 85747033 A  ERROR'
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:49:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:49:07 [DEBUG   ] spawn_process: at 2018-11-26 23:49:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:49:18 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 23:49:38 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-26 23:49:44 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:49:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:49:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:49:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:49:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:49:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:49:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:49:45 [DEBUG   ] Finished updating board flags in 1.451 seconds
2018-11-26 23:49:46 [DEBUG   ] Got command BAM from MCS: ref #637825
2018-11-26 23:49:46 [INFO    ] Finished processing BAM request
2018-11-26 23:49:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:49:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637825   8 58448 85786025 A  ERROR'
2018-11-26 23:49:46 [DEBUG   ] Got command BAM from MCS: ref #637826
2018-11-26 23:49:46 [INFO    ] Finished processing BAM request
2018-11-26 23:49:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:49:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637826   8 58448 85786029 A  ERROR'
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:49:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:49:46 [DEBUG   ] spawn_process: at 2018-11-26 23:49:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:49:59 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2018-11-26 23:50:19 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 23:50:25 [DEBUG   ] Got command BAM from MCS: ref #637827
2018-11-26 23:50:25 [INFO    ] Finished processing BAM request
2018-11-26 23:50:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:50:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637827   8 58448 85825029 A  ERROR'
2018-11-26 23:50:25 [DEBUG   ] Got command BAM from MCS: ref #637828
2018-11-26 23:50:25 [INFO    ] Finished processing BAM request
2018-11-26 23:50:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:50:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637828   8 58448 85825033 A  ERROR'
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:50:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:50:25 [DEBUG   ] spawn_process: at 2018-11-26 23:50:25 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:50:39 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-26 23:50:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa10, chip 4
2018-11-26 23:50:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-26 23:50:44 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:50:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:50:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:50:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:50:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:50:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:50:45 [DEBUG   ] Finished updating board flags in 1.353 seconds
2018-11-26 23:50:48 [DEBUG   ] Got command RPT from MCS: ref #637829
2018-11-26 23:50:48 [DEBUG   ] summary = ERROR
2018-11-26 23:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637829  13 58448 85848845 A  ERRORERROR'
2018-11-26 23:50:48 [DEBUG   ] Got command RPT from MCS: ref #637830
2018-11-26 23:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637830  77 58448 85848846 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:50:59 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-26 23:51:04 [DEBUG   ] Got command BAM from MCS: ref #637846
2018-11-26 23:51:04 [INFO    ] Finished processing BAM request
2018-11-26 23:51:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:51:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637846   8 58448 85864033 A  ERROR'
2018-11-26 23:51:04 [DEBUG   ] Got command BAM from MCS: ref #637847
2018-11-26 23:51:04 [INFO    ] Finished processing BAM request
2018-11-26 23:51:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:51:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637847   8 58448 85864037 A  ERROR'
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:51:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:51:04 [DEBUG   ] spawn_process: at 2018-11-26 23:51:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:51:19 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-26 23:51:39 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-26 23:51:43 [DEBUG   ] Got command BAM from MCS: ref #637849
2018-11-26 23:51:43 [INFO    ] Finished processing BAM request
2018-11-26 23:51:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:51:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637849   8 58448 85903029 A  ERROR'
2018-11-26 23:51:43 [DEBUG   ] Got command BAM from MCS: ref #637848
2018-11-26 23:51:43 [INFO    ] Finished processing BAM request
2018-11-26 23:51:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:51:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637848   8 58448 85903033 A  ERROR'
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:51:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:51:43 [DEBUG   ] spawn_process: at 2018-11-26 23:51:43 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:51:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:51:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-26 23:51:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:51:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:51:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:51:45 [WARNING ] (13, 1, 'bot')
2018-11-26 23:51:45 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:51:45 [DEBUG   ] Finished updating board flags in 1.357 seconds
2018-11-26 23:51:59 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-26 23:52:19 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-26 23:52:22 [DEBUG   ] Got command BAM from MCS: ref #637855
2018-11-26 23:52:22 [INFO    ] Finished processing BAM request
2018-11-26 23:52:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:52:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637855   8 58448 85942029 A  ERROR'
2018-11-26 23:52:22 [DEBUG   ] Got command BAM from MCS: ref #637854
2018-11-26 23:52:22 [INFO    ] Finished processing BAM request
2018-11-26 23:52:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:52:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637854   8 58448 85942033 A  ERROR'
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:52:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:52:22 [DEBUG   ] spawn_process: at 2018-11-26 23:52:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:52:39 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 23:52:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:52:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:52:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:52:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:52:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:52:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:52:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:52:46 [DEBUG   ] Finished updating board flags in 2.055 seconds
2018-11-26 23:52:52 [DEBUG   ] Got command RPT from MCS: ref #637858
2018-11-26 23:52:52 [DEBUG   ] summary = ERROR
2018-11-26 23:52:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637858  13 58448 85972121 A  ERRORERROR'
2018-11-26 23:52:52 [DEBUG   ] Got command RPT from MCS: ref #637859
2018-11-26 23:52:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:52:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637859  77 58448 85972313 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:52:52 [DEBUG   ] Got command RPT from MCS: ref #637860
2018-11-26 23:52:52 [DEBUG   ] summary = ERROR
2018-11-26 23:52:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637860  13 58448 85972517 A  ERRORERROR'
2018-11-26 23:52:52 [DEBUG   ] Got command RPT from MCS: ref #637862
2018-11-26 23:52:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:52:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637862  77 58448 85972721 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:52:52 [DEBUG   ] Got command RPT from MCS: ref #637863
2018-11-26 23:52:52 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:52:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637863  12 58448 85972925 A  ERRORBžff'
2018-11-26 23:52:59 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-26 23:53:01 [DEBUG   ] Got command BAM from MCS: ref #637868
2018-11-26 23:53:01 [INFO    ] Finished processing BAM request
2018-11-26 23:53:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:53:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637868   8 58448 85981025 A  ERROR'
2018-11-26 23:53:01 [DEBUG   ] Got command BAM from MCS: ref #637869
2018-11-26 23:53:01 [INFO    ] Finished processing BAM request
2018-11-26 23:53:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:53:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637869   8 58448 85981029 A  ERROR'
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:53:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:53:01 [DEBUG   ] spawn_process: at 2018-11-26 23:53:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:53:19 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-26 23:53:39 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-26 23:53:40 [DEBUG   ] Got command BAM from MCS: ref #637870
2018-11-26 23:53:40 [INFO    ] Finished processing BAM request
2018-11-26 23:53:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:53:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637870   8 58448 86020029 A  ERROR'
2018-11-26 23:53:40 [DEBUG   ] Got command BAM from MCS: ref #637871
2018-11-26 23:53:40 [INFO    ] Finished processing BAM request
2018-11-26 23:53:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:53:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637871   8 58448 86020033 A  ERROR'
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:53:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:53:40 [DEBUG   ] spawn_process: at 2018-11-26 23:53:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:53:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:53:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:53:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-26 23:53:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:53:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:53:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:53:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:53:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:53:46 [DEBUG   ] Finished updating board flags in 2.001 seconds
2018-11-26 23:54:00 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-26 23:54:19 [DEBUG   ] Got command BAM from MCS: ref #637873
2018-11-26 23:54:19 [INFO    ] Finished processing BAM request
2018-11-26 23:54:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:54:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637873   8 58448 86059025 A  ERROR'
2018-11-26 23:54:19 [DEBUG   ] Got command BAM from MCS: ref #637872
2018-11-26 23:54:19 [INFO    ] Finished processing BAM request
2018-11-26 23:54:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:54:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637872   8 58448 86059029 A  ERROR'
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:54:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:54:19 [DEBUG   ] spawn_process: at 2018-11-26 23:54:19 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:54:20 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 23:54:40 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-26 23:54:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:54:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-26 23:54:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:54:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:54:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:54:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:54:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:54:46 [DEBUG   ] Finished updating board flags in 1.997 seconds
2018-11-26 23:54:58 [DEBUG   ] Got command BAM from MCS: ref #637874
2018-11-26 23:54:58 [INFO    ] Finished processing BAM request
2018-11-26 23:54:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:54:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637874   8 58448 86098025 A  ERROR'
2018-11-26 23:54:58 [DEBUG   ] Got command BAM from MCS: ref #637875
2018-11-26 23:54:58 [INFO    ] Finished processing BAM request
2018-11-26 23:54:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:54:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637875   8 58448 86098029 A  ERROR'
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:54:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:54:58 [DEBUG   ] spawn_process: at 2018-11-26 23:54:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:55:00 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-26 23:55:20 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-26 23:55:37 [DEBUG   ] Got command BAM from MCS: ref #637880
2018-11-26 23:55:37 [INFO    ] Finished processing BAM request
2018-11-26 23:55:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:55:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637880   8 58448 86137033 A  ERROR'
2018-11-26 23:55:37 [DEBUG   ] Got command BAM from MCS: ref #637881
2018-11-26 23:55:37 [INFO    ] Finished processing BAM request
2018-11-26 23:55:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:55:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637881   8 58448 86137037 A  ERROR'
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-26 23:55:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-26 23:55:37 [DEBUG   ] spawn_process: at 2018-11-26 23:55:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-26 23:55:40 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2018-11-26 23:55:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:55:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:55:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:55:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:55:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:55:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:55:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:55:46 [DEBUG   ] Finished updating board flags in 1.803 seconds
2018-11-26 23:55:48 [DEBUG   ] Got command RPT from MCS: ref #637882
2018-11-26 23:55:48 [DEBUG   ] summary = ERROR
2018-11-26 23:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637882  13 58448 86148849 A  ERRORERROR'
2018-11-26 23:55:48 [DEBUG   ] Got command RPT from MCS: ref #637883
2018-11-26 23:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637883  77 58448 86148850 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:56:00 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 23:56:16 [DEBUG   ] Got command BAM from MCS: ref #637900
2018-11-26 23:56:16 [INFO    ] Finished processing BAM request
2018-11-26 23:56:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:56:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637900   8 58448 86176033 A  ERROR'
2018-11-26 23:56:16 [DEBUG   ] Got command BAM from MCS: ref #637899
2018-11-26 23:56:16 [INFO    ] Finished processing BAM request
2018-11-26 23:56:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:56:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637899   8 58448 86176037 A  ERROR'
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-26 23:56:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-26 23:56:16 [DEBUG   ] spawn_process: at 2018-11-26 23:56:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-26 23:56:20 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-26 23:56:40 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-26 23:56:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:56:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-26 23:56:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:56:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:56:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:56:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:56:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:56:46 [DEBUG   ] Finished updating board flags in 1.799 seconds
2018-11-26 23:56:55 [DEBUG   ] Got command BAM from MCS: ref #637902
2018-11-26 23:56:55 [INFO    ] Finished processing BAM request
2018-11-26 23:56:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:56:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637902   8 58448 86215025 A  ERROR'
2018-11-26 23:56:55 [DEBUG   ] Got command BAM from MCS: ref #637901
2018-11-26 23:56:55 [INFO    ] Finished processing BAM request
2018-11-26 23:56:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:56:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637901   8 58448 86215029 A  ERROR'
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-26 23:56:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-26 23:56:55 [DEBUG   ] spawn_process: at 2018-11-26 23:56:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-26 23:57:00 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-26 23:57:20 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-26 23:57:34 [DEBUG   ] Got command BAM from MCS: ref #637903
2018-11-26 23:57:34 [INFO    ] Finished processing BAM request
2018-11-26 23:57:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:57:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637903   8 58448 86254029 A  ERROR'
2018-11-26 23:57:34 [DEBUG   ] Got command BAM from MCS: ref #637904
2018-11-26 23:57:34 [INFO    ] Finished processing BAM request
2018-11-26 23:57:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:57:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637904   8 58448 86254033 A  ERROR'
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:57:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:57:34 [DEBUG   ] spawn_process: at 2018-11-26 23:57:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:57:40 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-26 23:57:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:57:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-26 23:57:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:57:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:57:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:57:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:57:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:57:46 [DEBUG   ] Finished updating board flags in 1.614 seconds
2018-11-26 23:57:52 [DEBUG   ] Got command RPT from MCS: ref #637905
2018-11-26 23:57:52 [DEBUG   ] summary = ERROR
2018-11-26 23:57:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637905  13 58448 86272201 A  ERRORERROR'
2018-11-26 23:57:52 [DEBUG   ] Got command RPT from MCS: ref #637906
2018-11-26 23:57:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:57:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637906  77 58448 86272405 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:57:52 [DEBUG   ] Got command RPT from MCS: ref #637907
2018-11-26 23:57:52 [DEBUG   ] summary = ERROR
2018-11-26 23:57:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637907  13 58448 86272609 A  ERRORERROR'
2018-11-26 23:57:52 [DEBUG   ] Got command RPT from MCS: ref #637908
2018-11-26 23:57:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-26 23:57:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637908  77 58448 86272801 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-26 23:57:53 [DEBUG   ] Got command RPT from MCS: ref #637909
2018-11-26 23:57:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-26 23:57:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637909  12 58448 86273005 A  ERRORBŸff'
2018-11-26 23:58:01 [DEBUG   ] Finished updating ant stats in 1.599 seconds
2018-11-26 23:58:13 [DEBUG   ] Got command BAM from MCS: ref #637915
2018-11-26 23:58:13 [INFO    ] Finished processing BAM request
2018-11-26 23:58:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:58:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637915   8 58448 86293025 A  ERROR'
2018-11-26 23:58:13 [DEBUG   ] Got command BAM from MCS: ref #637914
2018-11-26 23:58:13 [INFO    ] Finished processing BAM request
2018-11-26 23:58:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:58:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637914   8 58448 86293029 A  ERROR'
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-26 23:58:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-26 23:58:13 [DEBUG   ] spawn_process: at 2018-11-26 23:58:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-26 23:58:20 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-26 23:58:41 [DEBUG   ] Finished updating ant stats in 1.405 seconds
2018-11-26 23:58:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:58:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-26 23:58:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:58:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:58:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:58:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:58:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:58:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:58:46 [DEBUG   ] Finished updating board flags in 1.614 seconds
2018-11-26 23:58:52 [DEBUG   ] Got command BAM from MCS: ref #637917
2018-11-26 23:58:52 [INFO    ] Finished processing BAM request
2018-11-26 23:58:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:58:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637917   8 58448 86332029 A  ERROR'
2018-11-26 23:58:52 [DEBUG   ] Got command BAM from MCS: ref #637916
2018-11-26 23:58:52 [INFO    ] Finished processing BAM request
2018-11-26 23:58:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:58:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637916   8 58448 86332033 A  ERROR'
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:58:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:58:52 [DEBUG   ] spawn_process: at 2018-11-26 23:58:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:59:01 [DEBUG   ] Finished updating ant stats in 1.392 seconds
2018-11-26 23:59:21 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-26 23:59:31 [DEBUG   ] Got command BAM from MCS: ref #637919
2018-11-26 23:59:31 [INFO    ] Finished processing BAM request
2018-11-26 23:59:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:59:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637919   8 58448 86371029 A  ERROR'
2018-11-26 23:59:31 [DEBUG   ] Got command BAM from MCS: ref #637918
2018-11-26 23:59:31 [INFO    ] Finished processing BAM request
2018-11-26 23:59:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-26 23:59:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637918   8 58448 86371033 A  ERROR'
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-26 23:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-26 23:59:31 [DEBUG   ] spawn_process: at 2018-11-26 23:59:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-26 23:59:41 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-26 23:59:45 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-26 23:59:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-26 23:59:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:59:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:59:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:59:46 [WARNING ] (13, 1, 'bot')
2018-11-26 23:59:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-26 23:59:46 [DEBUG   ] Finished updating board flags in 1.639 seconds
2018-11-27 00:00:01 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 00:00:10 [DEBUG   ] Got command BAM from MCS: ref #637920
2018-11-27 00:00:10 [INFO    ] Finished processing BAM request
2018-11-27 00:00:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:00:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637920   8 58449    10025 A  ERROR'
2018-11-27 00:00:10 [DEBUG   ] Got command BAM from MCS: ref #637921
2018-11-27 00:00:10 [INFO    ] Finished processing BAM request
2018-11-27 00:00:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:00:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637921   8 58449    10029 A  ERROR'
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:00:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:00:10 [DEBUG   ] spawn_process: at 2018-11-27 00:00:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:00:21 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 00:00:41 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 00:00:45 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:00:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:00:46 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 00:00:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:00:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:00:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:00:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:00:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:00:46 [DEBUG   ] Finished updating board flags in 1.568 seconds
2018-11-27 00:00:48 [DEBUG   ] Got command RPT from MCS: ref #637924
2018-11-27 00:00:48 [DEBUG   ] summary = ERROR
2018-11-27 00:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637924  13 58449    48849 A  ERRORERROR'
2018-11-27 00:00:48 [DEBUG   ] Got command RPT from MCS: ref #637925
2018-11-27 00:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637925  77 58449    48850 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:00:49 [DEBUG   ] Got command BAM from MCS: ref #637922
2018-11-27 00:00:49 [INFO    ] Finished processing BAM request
2018-11-27 00:00:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:00:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637922   8 58449    49033 A  ERROR'
2018-11-27 00:00:49 [DEBUG   ] Got command BAM from MCS: ref #637923
2018-11-27 00:00:49 [INFO    ] Finished processing BAM request
2018-11-27 00:00:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:00:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637923   8 58449    49037 A  ERROR'
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:00:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:00:49 [DEBUG   ] spawn_process: at 2018-11-27 00:00:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:01:01 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2018-11-27 00:01:21 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 00:01:28 [DEBUG   ] Got command BAM from MCS: ref #637945
2018-11-27 00:01:28 [INFO    ] Finished processing BAM request
2018-11-27 00:01:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:01:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637945   8 58449    88029 A  ERROR'
2018-11-27 00:01:28 [DEBUG   ] Got command BAM from MCS: ref #637946
2018-11-27 00:01:28 [INFO    ] Finished processing BAM request
2018-11-27 00:01:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:01:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637946   8 58449    88033 A  ERROR'
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:01:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:01:28 [DEBUG   ] spawn_process: at 2018-11-27 00:01:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:01:41 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 00:01:45 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:01:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 00:01:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:01:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:01:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:01:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:01:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:01:46 [DEBUG   ] Finished updating board flags in 1.469 seconds
2018-11-27 00:02:01 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2018-11-27 00:02:07 [DEBUG   ] Got command BAM from MCS: ref #637948
2018-11-27 00:02:07 [INFO    ] Finished processing BAM request
2018-11-27 00:02:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:02:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637948   8 58449   127033 A  ERROR'
2018-11-27 00:02:07 [DEBUG   ] Got command BAM from MCS: ref #637947
2018-11-27 00:02:07 [INFO    ] Finished processing BAM request
2018-11-27 00:02:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:02:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637947   8 58449   127037 A  ERROR'
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:02:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:02:07 [DEBUG   ] spawn_process: at 2018-11-27 00:02:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:02:21 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-27 00:02:41 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 00:02:45 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:02:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 00:02:46 [DEBUG   ] Got command BAM from MCS: ref #637952
2018-11-27 00:02:46 [INFO    ] Finished processing BAM request
2018-11-27 00:02:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:02:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637952   8 58449   166029 A  ERROR'
2018-11-27 00:02:46 [DEBUG   ] Got command BAM from MCS: ref #637951
2018-11-27 00:02:46 [INFO    ] Finished processing BAM request
2018-11-27 00:02:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:02:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637951   8 58449   166033 A  ERROR'
2018-11-27 00:02:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:02:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:02:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:02:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:02:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:02:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:02:46 [DEBUG   ] Finished updating board flags in 1.474 seconds
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:02:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:02:46 [DEBUG   ] spawn_process: at 2018-11-27 00:02:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:02:52 [DEBUG   ] Got command RPT from MCS: ref #637953
2018-11-27 00:02:52 [DEBUG   ] summary = ERROR
2018-11-27 00:02:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637953  13 58449   172285 A  ERRORERROR'
2018-11-27 00:02:52 [DEBUG   ] Got command RPT from MCS: ref #637954
2018-11-27 00:02:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:02:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637954  77 58449   172489 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:02:52 [DEBUG   ] Got command RPT from MCS: ref #637955
2018-11-27 00:02:52 [DEBUG   ] summary = ERROR
2018-11-27 00:02:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637955  13 58449   172693 A  ERRORERROR'
2018-11-27 00:02:52 [DEBUG   ] Got command RPT from MCS: ref #637957
2018-11-27 00:02:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:02:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637957  77 58449   172897 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:02:53 [DEBUG   ] Got command RPT from MCS: ref #637958
2018-11-27 00:02:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:02:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637958  12 58449   173089 A  ERRORBŸff'
2018-11-27 00:03:01 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-27 00:03:21 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 00:03:25 [DEBUG   ] Got command BAM from MCS: ref #637963
2018-11-27 00:03:25 [INFO    ] Finished processing BAM request
2018-11-27 00:03:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:03:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637963   8 58449   205029 A  ERROR'
2018-11-27 00:03:25 [DEBUG   ] Got command BAM from MCS: ref #637964
2018-11-27 00:03:25 [INFO    ] Finished processing BAM request
2018-11-27 00:03:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:03:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637964   8 58449   205033 A  ERROR'
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:03:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:03:25 [DEBUG   ] spawn_process: at 2018-11-27 00:03:25 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:03:41 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 00:03:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:03:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:03:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:03:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:03:46 [WARNING ] (13, 1, 'bot')
2018-11-27 00:03:46 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:03:46 [DEBUG   ] Finished updating board flags in 1.475 seconds
2018-11-27 00:04:01 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2018-11-27 00:04:04 [DEBUG   ] Got command BAM from MCS: ref #637969
2018-11-27 00:04:04 [INFO    ] Finished processing BAM request
2018-11-27 00:04:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:04:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637969   8 58449   244025 A  ERROR'
2018-11-27 00:04:04 [DEBUG   ] Got command BAM from MCS: ref #637968
2018-11-27 00:04:04 [INFO    ] Finished processing BAM request
2018-11-27 00:04:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:04:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637968   8 58449   244029 A  ERROR'
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:04:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:04:04 [DEBUG   ] spawn_process: at 2018-11-27 00:04:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:04:21 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 00:04:42 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2018-11-27 00:04:43 [DEBUG   ] Got command BAM from MCS: ref #637971
2018-11-27 00:04:43 [INFO    ] Finished processing BAM request
2018-11-27 00:04:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:04:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637971   8 58449   283033 A  ERROR'
2018-11-27 00:04:43 [DEBUG   ] Got command BAM from MCS: ref #637972
2018-11-27 00:04:43 [INFO    ] Finished processing BAM request
2018-11-27 00:04:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:04:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637972   8 58449   283037 A  ERROR'
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:04:43 [DEBUG   ] spawn_process: at 2018-11-27 00:04:43 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:04:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:04:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 00:04:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:04:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:04:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:04:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:04:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:04:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:04:47 [DEBUG   ] Finished updating board flags in 2.192 seconds
2018-11-27 00:05:01 [DEBUG   ] Finished updating ant stats in 0.406 seconds
2018-11-27 00:05:22 [DEBUG   ] Got command BAM from MCS: ref #637973
2018-11-27 00:05:22 [INFO    ] Finished processing BAM request
2018-11-27 00:05:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:05:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637973   8 58449   322033 A  ERROR'
2018-11-27 00:05:22 [DEBUG   ] Got command BAM from MCS: ref #637974
2018-11-27 00:05:22 [INFO    ] Finished processing BAM request
2018-11-27 00:05:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:05:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637974   8 58449   322037 A  ERROR'
2018-11-27 00:05:22 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:05:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:05:22 [DEBUG   ] spawn_process: at 2018-11-27 00:05:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:05:42 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 00:05:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:05:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:05:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:05:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:05:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:05:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:05:47 [DEBUG   ] Finished updating board flags in 2.017 seconds
2018-11-27 00:05:48 [DEBUG   ] Got command RPT from MCS: ref #637975
2018-11-27 00:05:48 [DEBUG   ] summary = ERROR
2018-11-27 00:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637975  13 58449   348849 A  ERRORERROR'
2018-11-27 00:05:48 [DEBUG   ] Got command RPT from MCS: ref #637976
2018-11-27 00:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   637976  77 58449   348850 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:06:01 [DEBUG   ] Got command BAM from MCS: ref #637993
2018-11-27 00:06:01 [INFO    ] Finished processing BAM request
2018-11-27 00:06:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:06:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637993   8 58449   361033 A  ERROR'
2018-11-27 00:06:01 [DEBUG   ] Got command BAM from MCS: ref #637992
2018-11-27 00:06:01 [INFO    ] Finished processing BAM request
2018-11-27 00:06:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:06:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637992   8 58449   361037 A  ERROR'
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:06:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:06:01 [DEBUG   ] spawn_process: at 2018-11-27 00:06:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:06:02 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 00:06:22 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 00:06:40 [DEBUG   ] Got command BAM from MCS: ref #637994
2018-11-27 00:06:40 [INFO    ] Finished processing BAM request
2018-11-27 00:06:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:06:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637994   8 58449   400025 A  ERROR'
2018-11-27 00:06:40 [DEBUG   ] Got command BAM from MCS: ref #637995
2018-11-27 00:06:40 [INFO    ] Finished processing BAM request
2018-11-27 00:06:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:06:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   637995   8 58449   400029 A  ERROR'
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:06:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:06:40 [DEBUG   ] spawn_process: at 2018-11-27 00:06:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:06:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 00:06:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:06:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 00:06:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:06:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:06:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:06:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:06:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:06:47 [DEBUG   ] Finished updating board flags in 1.936 seconds
2018-11-27 00:07:02 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 00:07:20 [DEBUG   ] Got command BAM from MCS: ref #638001
2018-11-27 00:07:20 [INFO    ] Finished processing BAM request
2018-11-27 00:07:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:07:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638001   8 58449   440029 A  ERROR'
2018-11-27 00:07:20 [DEBUG   ] Got command BAM from MCS: ref #638000
2018-11-27 00:07:20 [INFO    ] Finished processing BAM request
2018-11-27 00:07:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:07:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638000   8 58449   440033 A  ERROR'
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:07:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:07:20 [DEBUG   ] spawn_process: at 2018-11-27 00:07:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:07:22 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 00:07:42 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-27 00:07:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:07:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 00:07:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:07:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:07:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:07:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:07:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:07:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:07:47 [DEBUG   ] Finished updating board flags in 1.815 seconds
2018-11-27 00:07:52 [DEBUG   ] Got command RPT from MCS: ref #638002
2018-11-27 00:07:52 [DEBUG   ] summary = ERROR
2018-11-27 00:07:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638002  13 58449   472381 A  ERRORERROR'
2018-11-27 00:07:52 [DEBUG   ] Got command RPT from MCS: ref #638003
2018-11-27 00:07:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:07:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638003  77 58449   472573 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:07:52 [DEBUG   ] Got command RPT from MCS: ref #638004
2018-11-27 00:07:52 [DEBUG   ] summary = ERROR
2018-11-27 00:07:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638004  13 58449   472777 A  ERRORERROR'
2018-11-27 00:07:52 [DEBUG   ] Got command RPT from MCS: ref #638005
2018-11-27 00:07:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:07:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638005  77 58449   472981 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:07:53 [DEBUG   ] Got command RPT from MCS: ref #638006
2018-11-27 00:07:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:07:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638006  12 58449   473185 A  ERRORBŸff'
2018-11-27 00:08:00 [DEBUG   ] Got command BAM from MCS: ref #638008
2018-11-27 00:08:00 [INFO    ] Finished processing BAM request
2018-11-27 00:08:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638008   8 58449   480033 A  ERROR'
2018-11-27 00:08:00 [DEBUG   ] Got command BAM from MCS: ref #638007
2018-11-27 00:08:00 [INFO    ] Finished processing BAM request
2018-11-27 00:08:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638007   8 58449   480037 A  ERROR'
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:08:00 [DEBUG   ] spawn_process: at 2018-11-27 00:08:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:08:02 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 00:08:22 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 00:08:40 [DEBUG   ] Got command BAM from MCS: ref #638009
2018-11-27 00:08:40 [INFO    ] Finished processing BAM request
2018-11-27 00:08:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:08:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638009   8 58449   520025 A  ERROR'
2018-11-27 00:08:40 [DEBUG   ] Got command BAM from MCS: ref #638010
2018-11-27 00:08:40 [INFO    ] Finished processing BAM request
2018-11-27 00:08:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:08:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638010   8 58449   520029 A  ERROR'
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:08:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:08:40 [DEBUG   ] spawn_process: at 2018-11-27 00:08:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:08:42 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-27 00:08:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:08:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:08:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:08:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:08:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:08:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:08:47 [DEBUG   ] Finished updating board flags in 1.775 seconds
2018-11-27 00:09:02 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-27 00:09:20 [DEBUG   ] Got command BAM from MCS: ref #638011
2018-11-27 00:09:20 [INFO    ] Finished processing BAM request
2018-11-27 00:09:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:09:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638011   8 58449   560029 A  ERROR'
2018-11-27 00:09:20 [DEBUG   ] Got command BAM from MCS: ref #638012
2018-11-27 00:09:20 [INFO    ] Finished processing BAM request
2018-11-27 00:09:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:09:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638012   8 58449   560033 A  ERROR'
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:09:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:09:20 [DEBUG   ] spawn_process: at 2018-11-27 00:09:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:09:22 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 00:09:42 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 00:09:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:09:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:09:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:09:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:09:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:09:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:09:47 [DEBUG   ] Finished updating board flags in 1.617 seconds
2018-11-27 00:10:00 [DEBUG   ] Got command BAM from MCS: ref #638014
2018-11-27 00:10:00 [INFO    ] Finished processing BAM request
2018-11-27 00:10:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:10:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638014   8 58449   600025 A  ERROR'
2018-11-27 00:10:00 [DEBUG   ] Got command BAM from MCS: ref #638013
2018-11-27 00:10:00 [INFO    ] Finished processing BAM request
2018-11-27 00:10:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:10:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638013   8 58449   600029 A  ERROR'
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:10:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:10:00 [DEBUG   ] spawn_process: at 2018-11-27 00:10:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:10:02 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2018-11-27 00:10:22 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2018-11-27 00:10:40 [DEBUG   ] Got command BAM from MCS: ref #638020
2018-11-27 00:10:40 [INFO    ] Finished processing BAM request
2018-11-27 00:10:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:10:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638020   8 58449   640025 A  ERROR'
2018-11-27 00:10:40 [DEBUG   ] Got command BAM from MCS: ref #638019
2018-11-27 00:10:40 [INFO    ] Finished processing BAM request
2018-11-27 00:10:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:10:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638019   8 58449   640029 A  ERROR'
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:10:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:10:40 [DEBUG   ] spawn_process: at 2018-11-27 00:10:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:10:42 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 00:10:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:10:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:10:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:10:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:10:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:10:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:10:47 [DEBUG   ] Finished updating board flags in 1.607 seconds
2018-11-27 00:10:48 [DEBUG   ] Got command RPT from MCS: ref #638021
2018-11-27 00:10:48 [DEBUG   ] summary = ERROR
2018-11-27 00:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638021  13 58449   648849 A  ERRORERROR'
2018-11-27 00:10:48 [DEBUG   ] Got command RPT from MCS: ref #638022
2018-11-27 00:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638022  77 58449   648850 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:11:03 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2018-11-27 00:11:20 [DEBUG   ] Got command BAM from MCS: ref #638038
2018-11-27 00:11:20 [INFO    ] Finished processing BAM request
2018-11-27 00:11:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:11:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638038   8 58449   680029 A  ERROR'
2018-11-27 00:11:20 [DEBUG   ] Got command BAM from MCS: ref #638039
2018-11-27 00:11:20 [INFO    ] Finished processing BAM request
2018-11-27 00:11:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:11:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638039   8 58449   680033 A  ERROR'
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:11:20 [DEBUG   ] spawn_process: at 2018-11-27 00:11:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:11:22 [DEBUG   ] Finished updating ant stats in 0.405 seconds
2018-11-27 00:11:43 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2018-11-27 00:11:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 00:11:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:11:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:11:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:11:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:11:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:11:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:11:47 [DEBUG   ] Finished updating board flags in 1.590 seconds
2018-11-27 00:12:00 [DEBUG   ] Got command BAM from MCS: ref #638040
2018-11-27 00:12:00 [INFO    ] Finished processing BAM request
2018-11-27 00:12:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:12:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638040   8 58449   720029 A  ERROR'
2018-11-27 00:12:00 [DEBUG   ] Got command BAM from MCS: ref #638041
2018-11-27 00:12:00 [INFO    ] Finished processing BAM request
2018-11-27 00:12:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:12:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638041   8 58449   720033 A  ERROR'
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:12:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:12:00 [DEBUG   ] spawn_process: at 2018-11-27 00:12:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:12:03 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 00:12:23 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 00:12:40 [DEBUG   ] Got command BAM from MCS: ref #638044
2018-11-27 00:12:40 [INFO    ] Finished processing BAM request
2018-11-27 00:12:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:12:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638044   8 58449   760033 A  ERROR'
2018-11-27 00:12:40 [DEBUG   ] Got command BAM from MCS: ref #638045
2018-11-27 00:12:40 [INFO    ] Finished processing BAM request
2018-11-27 00:12:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:12:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638045   8 58449   760037 A  ERROR'
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:12:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:12:40 [DEBUG   ] spawn_process: at 2018-11-27 00:12:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:12:43 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 00:12:46 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:12:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:12:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:12:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:12:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:12:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:12:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:12:47 [DEBUG   ] Finished updating board flags in 1.403 seconds
2018-11-27 00:12:52 [DEBUG   ] Got command RPT from MCS: ref #638046
2018-11-27 00:12:52 [DEBUG   ] summary = ERROR
2018-11-27 00:12:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638046  13 58449   772461 A  ERRORERROR'
2018-11-27 00:12:52 [DEBUG   ] Got command RPT from MCS: ref #638047
2018-11-27 00:12:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:12:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638047  77 58449   772665 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:12:52 [DEBUG   ] Got command RPT from MCS: ref #638048
2018-11-27 00:12:52 [DEBUG   ] summary = ERROR
2018-11-27 00:12:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638048  13 58449   772869 A  ERRORERROR'
2018-11-27 00:12:53 [DEBUG   ] Got command RPT from MCS: ref #638050
2018-11-27 00:12:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:12:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638050  77 58449   773073 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:12:53 [DEBUG   ] Got command RPT from MCS: ref #638051
2018-11-27 00:12:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:12:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638051  12 58449   773265 A  ERRORBžff'
2018-11-27 00:13:03 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 00:13:20 [DEBUG   ] Got command BAM from MCS: ref #638061
2018-11-27 00:13:20 [INFO    ] Finished processing BAM request
2018-11-27 00:13:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:13:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638061   8 58449   800030 A  ERROR'
2018-11-27 00:13:20 [DEBUG   ] Got command BAM from MCS: ref #638060
2018-11-27 00:13:20 [INFO    ] Finished processing BAM request
2018-11-27 00:13:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:13:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638060   8 58449   800034 A  ERROR'
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:13:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:13:20 [DEBUG   ] spawn_process: at 2018-11-27 00:13:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:13:23 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 00:13:43 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 00:13:46 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:13:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 00:13:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:13:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:13:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:13:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:13:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:13:47 [DEBUG   ] Finished updating board flags in 1.398 seconds
2018-11-27 00:14:00 [DEBUG   ] Got command BAM from MCS: ref #638063
2018-11-27 00:14:00 [INFO    ] Finished processing BAM request
2018-11-27 00:14:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:14:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638063   8 58449   840026 A  ERROR'
2018-11-27 00:14:00 [DEBUG   ] Got command BAM from MCS: ref #638062
2018-11-27 00:14:00 [INFO    ] Finished processing BAM request
2018-11-27 00:14:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:14:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638062   8 58449   840030 A  ERROR'
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:14:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:14:00 [DEBUG   ] spawn_process: at 2018-11-27 00:14:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:14:03 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-27 00:14:23 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 00:14:40 [DEBUG   ] Got command BAM from MCS: ref #638065
2018-11-27 00:14:40 [INFO    ] Finished processing BAM request
2018-11-27 00:14:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:14:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638065   8 58449   880030 A  ERROR'
2018-11-27 00:14:40 [DEBUG   ] Got command BAM from MCS: ref #638064
2018-11-27 00:14:40 [INFO    ] Finished processing BAM request
2018-11-27 00:14:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:14:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638064   8 58449   880034 A  ERROR'
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:14:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:14:40 [DEBUG   ] spawn_process: at 2018-11-27 00:14:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:14:43 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 00:14:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:14:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:14:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:14:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:14:47 [WARNING ] (13, 1, 'bot')
2018-11-27 00:14:47 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:14:47 [DEBUG   ] Finished updating board flags in 1.343 seconds
2018-11-27 00:15:03 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-27 00:15:20 [DEBUG   ] Got command BAM from MCS: ref #638067
2018-11-27 00:15:20 [INFO    ] Finished processing BAM request
2018-11-27 00:15:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:15:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638067   8 58449   920033 A  ERROR'
2018-11-27 00:15:20 [DEBUG   ] Got command BAM from MCS: ref #638066
2018-11-27 00:15:20 [INFO    ] Finished processing BAM request
2018-11-27 00:15:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:15:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638066   8 58449   920037 A  ERROR'
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:15:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:15:20 [DEBUG   ] spawn_process: at 2018-11-27 00:15:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:15:23 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 00:15:43 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 00:15:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 00:15:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:15:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 00:15:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 00:15:48 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:15:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 00:15:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:15:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:15:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:15:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:15:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:15:48 [DEBUG   ] Finished updating board flags in 2.066 seconds
2018-11-27 00:15:48 [DEBUG   ] Got command RPT from MCS: ref #638068
2018-11-27 00:15:48 [DEBUG   ] summary = ERROR
2018-11-27 00:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638068  13 58449   948849 A  ERRORERROR'
2018-11-27 00:15:48 [DEBUG   ] Got command RPT from MCS: ref #638069
2018-11-27 00:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638069  77 58449   948850 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:16:00 [DEBUG   ] Got command BAM from MCS: ref #638085
2018-11-27 00:16:00 [INFO    ] Finished processing BAM request
2018-11-27 00:16:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:16:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638085   8 58449   960025 A  ERROR'
2018-11-27 00:16:00 [DEBUG   ] Got command BAM from MCS: ref #638086
2018-11-27 00:16:00 [INFO    ] Finished processing BAM request
2018-11-27 00:16:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:16:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638086   8 58449   960029 A  ERROR'
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:16:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:16:00 [DEBUG   ] spawn_process: at 2018-11-27 00:16:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:16:04 [DEBUG   ] Finished updating ant stats in 1.393 seconds
2018-11-27 00:16:24 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 00:16:40 [DEBUG   ] Got command BAM from MCS: ref #638091
2018-11-27 00:16:40 [INFO    ] Finished processing BAM request
2018-11-27 00:16:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:16:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638091   8 58449  1000025 A  ERROR'
2018-11-27 00:16:40 [DEBUG   ] Got command BAM from MCS: ref #638092
2018-11-27 00:16:40 [INFO    ] Finished processing BAM request
2018-11-27 00:16:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:16:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638092   8 58449  1000029 A  ERROR'
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:16:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:16:40 [DEBUG   ] spawn_process: at 2018-11-27 00:16:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:16:44 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 00:16:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:16:48 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 00:16:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:16:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:16:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:16:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:16:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:16:48 [DEBUG   ] Finished updating board flags in 2.012 seconds
2018-11-27 00:17:04 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 00:17:20 [DEBUG   ] Got command BAM from MCS: ref #638093
2018-11-27 00:17:20 [INFO    ] Finished processing BAM request
2018-11-27 00:17:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:17:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638093   8 58449  1040033 A  ERROR'
2018-11-27 00:17:20 [DEBUG   ] Got command BAM from MCS: ref #638094
2018-11-27 00:17:20 [INFO    ] Finished processing BAM request
2018-11-27 00:17:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:17:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638094   8 58449  1040037 A  ERROR'
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:17:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:17:20 [DEBUG   ] spawn_process: at 2018-11-27 00:17:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:17:24 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 00:17:44 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 00:17:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 00:17:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:17:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 00:17:48 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 00:17:48 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 00:17:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:17:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:17:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:17:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:17:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:17:48 [DEBUG   ] Finished updating board flags in 1.983 seconds
2018-11-27 00:17:52 [DEBUG   ] Got command RPT from MCS: ref #638095
2018-11-27 00:17:52 [DEBUG   ] summary = ERROR
2018-11-27 00:17:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638095  13 58449  1072553 A  ERRORERROR'
2018-11-27 00:17:52 [DEBUG   ] Got command RPT from MCS: ref #638096
2018-11-27 00:17:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:17:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638096  77 58449  1072757 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:17:52 [DEBUG   ] Got command RPT from MCS: ref #638097
2018-11-27 00:17:52 [DEBUG   ] summary = ERROR
2018-11-27 00:17:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638097  13 58449  1072961 A  ERRORERROR'
2018-11-27 00:17:53 [DEBUG   ] Got command RPT from MCS: ref #638098
2018-11-27 00:17:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:17:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638098  77 58449  1073153 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:17:53 [DEBUG   ] Got command RPT from MCS: ref #638099
2018-11-27 00:17:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:17:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638099  12 58449  1073357 A  ERRORB¡ff'
2018-11-27 00:18:00 [DEBUG   ] Got command BAM from MCS: ref #638100
2018-11-27 00:18:00 [INFO    ] Finished processing BAM request
2018-11-27 00:18:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:18:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638100   8 58449  1080033 A  ERROR'
2018-11-27 00:18:00 [DEBUG   ] Got command BAM from MCS: ref #638101
2018-11-27 00:18:00 [INFO    ] Finished processing BAM request
2018-11-27 00:18:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:18:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638101   8 58449  1080037 A  ERROR'
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:18:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:18:00 [DEBUG   ] spawn_process: at 2018-11-27 00:18:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:18:04 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 00:18:24 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 00:18:40 [DEBUG   ] Got command BAM from MCS: ref #638102
2018-11-27 00:18:40 [INFO    ] Finished processing BAM request
2018-11-27 00:18:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:18:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638102   8 58449  1120029 A  ERROR'
2018-11-27 00:18:40 [DEBUG   ] Got command BAM from MCS: ref #638103
2018-11-27 00:18:40 [INFO    ] Finished processing BAM request
2018-11-27 00:18:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:18:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638103   8 58449  1120033 A  ERROR'
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:18:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:18:40 [DEBUG   ] spawn_process: at 2018-11-27 00:18:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:18:44 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-27 00:18:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:18:48 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:18:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:18:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:18:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:18:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:18:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:18:48 [DEBUG   ] Finished updating board flags in 1.812 seconds
2018-11-27 00:19:04 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-27 00:19:20 [DEBUG   ] Got command BAM from MCS: ref #638108
2018-11-27 00:19:20 [INFO    ] Finished processing BAM request
2018-11-27 00:19:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:19:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638108   8 58449  1160041 A  ERROR'
2018-11-27 00:19:20 [DEBUG   ] Got command BAM from MCS: ref #638109
2018-11-27 00:19:20 [INFO    ] Finished processing BAM request
2018-11-27 00:19:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:19:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638109   8 58449  1160045 A  ERROR'
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:19:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:19:20 [DEBUG   ] spawn_process: at 2018-11-27 00:19:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:19:24 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 00:19:44 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 00:19:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 00:19:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:19:48 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:19:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:19:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:19:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:19:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:19:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:19:48 [DEBUG   ] Finished updating board flags in 1.793 seconds
2018-11-27 00:20:00 [DEBUG   ] Got command BAM from MCS: ref #638110
2018-11-27 00:20:00 [INFO    ] Finished processing BAM request
2018-11-27 00:20:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:20:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638110   8 58449  1200033 A  ERROR'
2018-11-27 00:20:00 [DEBUG   ] Got command BAM from MCS: ref #638111
2018-11-27 00:20:00 [INFO    ] Finished processing BAM request
2018-11-27 00:20:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:20:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638111   8 58449  1200037 A  ERROR'
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:20:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:20:00 [DEBUG   ] spawn_process: at 2018-11-27 00:20:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:20:04 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-27 00:20:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-27 00:20:40 [DEBUG   ] Got command BAM from MCS: ref #638112
2018-11-27 00:20:40 [INFO    ] Finished processing BAM request
2018-11-27 00:20:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:20:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638112   8 58449  1240029 A  ERROR'
2018-11-27 00:20:40 [DEBUG   ] Got command BAM from MCS: ref #638113
2018-11-27 00:20:40 [INFO    ] Finished processing BAM request
2018-11-27 00:20:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:20:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638113   8 58449  1240033 A  ERROR'
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:20:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:20:40 [DEBUG   ] spawn_process: at 2018-11-27 00:20:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:20:44 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 00:20:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:20:48 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:20:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 00:20:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:20:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:20:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:20:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:20:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:20:48 [DEBUG   ] Finished updating board flags in 1.675 seconds
2018-11-27 00:20:48 [DEBUG   ] Got command RPT from MCS: ref #638114
2018-11-27 00:20:48 [DEBUG   ] summary = ERROR
2018-11-27 00:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638114  13 58449  1248853 A  ERRORERROR'
2018-11-27 00:20:48 [DEBUG   ] Got command RPT from MCS: ref #638115
2018-11-27 00:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638115  77 58449  1248854 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:21:04 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 00:21:20 [DEBUG   ] Got command BAM from MCS: ref #638131
2018-11-27 00:21:20 [INFO    ] Finished processing BAM request
2018-11-27 00:21:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:21:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638131   8 58449  1280025 A  ERROR'
2018-11-27 00:21:20 [DEBUG   ] Got command BAM from MCS: ref #638132
2018-11-27 00:21:20 [INFO    ] Finished processing BAM request
2018-11-27 00:21:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:21:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638132   8 58449  1280029 A  ERROR'
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:21:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:21:20 [DEBUG   ] spawn_process: at 2018-11-27 00:21:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:21:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 00:21:44 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 00:21:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:21:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 00:21:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:21:48 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 00:21:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:21:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:21:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:21:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:21:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:21:48 [DEBUG   ] Finished updating board flags in 1.611 seconds
2018-11-27 00:22:00 [DEBUG   ] Got command BAM from MCS: ref #638134
2018-11-27 00:22:00 [INFO    ] Finished processing BAM request
2018-11-27 00:22:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:22:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638134   8 58449  1320029 A  ERROR'
2018-11-27 00:22:00 [DEBUG   ] Got command BAM from MCS: ref #638133
2018-11-27 00:22:00 [INFO    ] Finished processing BAM request
2018-11-27 00:22:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:22:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638133   8 58449  1320033 A  ERROR'
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:22:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:22:00 [DEBUG   ] spawn_process: at 2018-11-27 00:22:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:22:05 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 00:22:25 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-27 00:22:40 [DEBUG   ] Got command BAM from MCS: ref #638141
2018-11-27 00:22:40 [INFO    ] Finished processing BAM request
2018-11-27 00:22:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:22:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638141   8 58449  1360033 A  ERROR'
2018-11-27 00:22:40 [DEBUG   ] Got command BAM from MCS: ref #638142
2018-11-27 00:22:40 [INFO    ] Finished processing BAM request
2018-11-27 00:22:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:22:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638142   8 58449  1360037 A  ERROR'
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:22:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:22:40 [DEBUG   ] spawn_process: at 2018-11-27 00:22:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:22:45 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 00:22:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:22:48 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:22:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:22:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:22:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:22:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:22:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:22:48 [DEBUG   ] Finished updating board flags in 1.522 seconds
2018-11-27 00:22:52 [DEBUG   ] Got command RPT from MCS: ref #638143
2018-11-27 00:22:52 [DEBUG   ] summary = ERROR
2018-11-27 00:22:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638143  13 58449  1372637 A  ERRORERROR'
2018-11-27 00:22:52 [DEBUG   ] Got command RPT from MCS: ref #638144
2018-11-27 00:22:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:22:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638144  77 58449  1372841 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:22:53 [DEBUG   ] Got command RPT from MCS: ref #638145
2018-11-27 00:22:53 [DEBUG   ] summary = ERROR
2018-11-27 00:22:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638145  13 58449  1373045 A  ERRORERROR'
2018-11-27 00:22:53 [DEBUG   ] Got command RPT from MCS: ref #638147
2018-11-27 00:22:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:22:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638147  77 58449  1373249 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:22:53 [DEBUG   ] Got command RPT from MCS: ref #638148
2018-11-27 00:22:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:22:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638148  12 58449  1373441 A  ERRORB¢ff'
2018-11-27 00:23:05 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 00:23:20 [DEBUG   ] Got command BAM from MCS: ref #638154
2018-11-27 00:23:20 [INFO    ] Finished processing BAM request
2018-11-27 00:23:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:23:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638154   8 58449  1400025 A  ERROR'
2018-11-27 00:23:20 [DEBUG   ] Got command BAM from MCS: ref #638153
2018-11-27 00:23:20 [INFO    ] Finished processing BAM request
2018-11-27 00:23:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:23:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638153   8 58449  1400029 A  ERROR'
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:23:20 [DEBUG   ] spawn_process: at 2018-11-27 00:23:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:23:25 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 00:23:45 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-27 00:23:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 00:23:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:23:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 00:23:48 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:23:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:23:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:23:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:23:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:23:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:23:48 [DEBUG   ] Finished updating board flags in 1.411 seconds
2018-11-27 00:24:00 [DEBUG   ] Got command BAM from MCS: ref #638155
2018-11-27 00:24:00 [INFO    ] Finished processing BAM request
2018-11-27 00:24:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:24:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638155   8 58449  1440029 A  ERROR'
2018-11-27 00:24:00 [DEBUG   ] Got command BAM from MCS: ref #638156
2018-11-27 00:24:00 [INFO    ] Finished processing BAM request
2018-11-27 00:24:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:24:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638156   8 58449  1440033 A  ERROR'
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:24:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:24:00 [DEBUG   ] spawn_process: at 2018-11-27 00:24:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:24:05 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 00:24:26 [DEBUG   ] Finished updating ant stats in 1.605 seconds
2018-11-27 00:24:41 [DEBUG   ] Got command BAM from MCS: ref #638158
2018-11-27 00:24:41 [INFO    ] Finished processing BAM request
2018-11-27 00:24:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:24:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638158   8 58449  1481033 A  ERROR'
2018-11-27 00:24:41 [DEBUG   ] Got command BAM from MCS: ref #638157
2018-11-27 00:24:41 [INFO    ] Finished processing BAM request
2018-11-27 00:24:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:24:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638157   8 58449  1481037 A  ERROR'
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:24:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:24:41 [DEBUG   ] spawn_process: at 2018-11-27 00:24:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:24:45 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 00:24:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:24:48 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 00:24:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:24:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:24:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:24:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:24:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:24:48 [DEBUG   ] Finished updating board flags in 1.402 seconds
2018-11-27 00:25:06 [DEBUG   ] Finished updating ant stats in 1.599 seconds
2018-11-27 00:25:22 [DEBUG   ] Got command BAM from MCS: ref #638163
2018-11-27 00:25:22 [INFO    ] Finished processing BAM request
2018-11-27 00:25:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:25:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638163   8 58449  1522029 A  ERROR'
2018-11-27 00:25:22 [DEBUG   ] Got command BAM from MCS: ref #638164
2018-11-27 00:25:22 [INFO    ] Finished processing BAM request
2018-11-27 00:25:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:25:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638164   8 58449  1522033 A  ERROR'
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:25:22 [DEBUG   ] spawn_process: at 2018-11-27 00:25:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:25:26 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2018-11-27 00:25:46 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2018-11-27 00:25:47 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 00:25:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:25:48 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 00:25:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:25:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:25:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:25:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:25:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:25:48 [DEBUG   ] Finished updating board flags in 1.403 seconds
2018-11-27 00:25:48 [DEBUG   ] Got command RPT from MCS: ref #638165
2018-11-27 00:25:48 [DEBUG   ] summary = ERROR
2018-11-27 00:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638165  13 58449  1548845 A  ERRORERROR'
2018-11-27 00:25:48 [DEBUG   ] Got command RPT from MCS: ref #638166
2018-11-27 00:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638166  77 58449  1548846 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:26:03 [DEBUG   ] Got command BAM from MCS: ref #638183
2018-11-27 00:26:03 [INFO    ] Finished processing BAM request
2018-11-27 00:26:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:26:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638183   8 58449  1563033 A  ERROR'
2018-11-27 00:26:03 [DEBUG   ] Got command BAM from MCS: ref #638182
2018-11-27 00:26:03 [INFO    ] Finished processing BAM request
2018-11-27 00:26:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:26:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638182   8 58449  1563037 A  ERROR'
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:26:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:26:03 [DEBUG   ] spawn_process: at 2018-11-27 00:26:03 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:26:06 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 00:26:26 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 00:26:44 [DEBUG   ] Got command BAM from MCS: ref #638185
2018-11-27 00:26:44 [INFO    ] Finished processing BAM request
2018-11-27 00:26:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:26:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638185   8 58449  1604033 A  ERROR'
2018-11-27 00:26:44 [DEBUG   ] Got command BAM from MCS: ref #638184
2018-11-27 00:26:44 [INFO    ] Finished processing BAM request
2018-11-27 00:26:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:26:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638184   8 58449  1604037 A  ERROR'
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:26:44 [DEBUG   ] spawn_process: at 2018-11-27 00:26:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:26:46 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 00:26:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 00:26:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:26:48 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 00:26:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:26:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:26:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:26:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:26:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:26:48 [DEBUG   ] Finished updating board flags in 1.413 seconds
2018-11-27 00:27:06 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-27 00:27:25 [DEBUG   ] Got command BAM from MCS: ref #638186
2018-11-27 00:27:25 [INFO    ] Finished processing BAM request
2018-11-27 00:27:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:27:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638186   8 58449  1645033 A  ERROR'
2018-11-27 00:27:25 [DEBUG   ] Got command BAM from MCS: ref #638187
2018-11-27 00:27:25 [INFO    ] Finished processing BAM request
2018-11-27 00:27:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:27:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638187   8 58449  1645037 A  ERROR'
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:27:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:27:25 [DEBUG   ] spawn_process: at 2018-11-27 00:27:25 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:27:26 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-27 00:27:46 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 00:27:47 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:27:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-27 00:27:48 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 00:27:48 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 00:27:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 00:27:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:27:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:27:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:27:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:27:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:27:48 [DEBUG   ] Finished updating board flags in 1.386 seconds
2018-11-27 00:27:52 [DEBUG   ] Got command RPT from MCS: ref #638188
2018-11-27 00:27:52 [DEBUG   ] summary = ERROR
2018-11-27 00:27:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638188  13 58449  1672729 A  ERRORERROR'
2018-11-27 00:27:52 [DEBUG   ] Got command RPT from MCS: ref #638189
2018-11-27 00:27:52 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:27:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638189  77 58449  1672933 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:27:53 [DEBUG   ] Got command RPT from MCS: ref #638190
2018-11-27 00:27:53 [DEBUG   ] summary = ERROR
2018-11-27 00:27:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638190  13 58449  1673125 A  ERRORERROR'
2018-11-27 00:27:53 [DEBUG   ] Got command RPT from MCS: ref #638191
2018-11-27 00:27:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:27:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638191  77 58449  1673329 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:27:53 [DEBUG   ] Got command RPT from MCS: ref #638192
2018-11-27 00:27:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:27:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638192  12 58449  1673533 A  ERRORB¡ff'
2018-11-27 00:28:06 [DEBUG   ] Got command BAM from MCS: ref #638198
2018-11-27 00:28:06 [INFO    ] Finished processing BAM request
2018-11-27 00:28:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:28:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638198   8 58449  1686033 A  ERROR'
2018-11-27 00:28:06 [DEBUG   ] Got command BAM from MCS: ref #638197
2018-11-27 00:28:06 [INFO    ] Finished processing BAM request
2018-11-27 00:28:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:28:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638197   8 58449  1686037 A  ERROR'
2018-11-27 00:28:06 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:28:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:28:06 [DEBUG   ] spawn_process: at 2018-11-27 00:28:06 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:28:26 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-27 00:28:46 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 00:28:47 [DEBUG   ] Got command BAM from MCS: ref #638199
2018-11-27 00:28:47 [INFO    ] Finished processing BAM request
2018-11-27 00:28:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:28:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638199   8 58449  1727033 A  ERROR'
2018-11-27 00:28:47 [DEBUG   ] Got command BAM from MCS: ref #638200
2018-11-27 00:28:47 [INFO    ] Finished processing BAM request
2018-11-27 00:28:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:28:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638200   8 58449  1727037 A  ERROR'
2018-11-27 00:28:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:28:47 [DEBUG   ] spawn_process: at 2018-11-27 00:28:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:28:48 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:28:48 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:28:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 00:28:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:28:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:28:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:28:48 [WARNING ] (13, 1, 'bot')
2018-11-27 00:28:48 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:28:48 [DEBUG   ] Finished updating board flags in 1.395 seconds
2018-11-27 00:29:06 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 00:29:26 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 00:29:28 [DEBUG   ] Got command BAM from MCS: ref #638201
2018-11-27 00:29:28 [INFO    ] Finished processing BAM request
2018-11-27 00:29:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:29:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638201   8 58449  1768033 A  ERROR'
2018-11-27 00:29:28 [DEBUG   ] Got command BAM from MCS: ref #638202
2018-11-27 00:29:28 [INFO    ] Finished processing BAM request
2018-11-27 00:29:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:29:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638202   8 58449  1768037 A  ERROR'
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:29:28 [DEBUG   ] spawn_process: at 2018-11-27 00:29:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:29:46 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-27 00:29:48 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:29:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 00:29:49 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 00:29:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:29:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:29:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:29:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:29:49 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:29:49 [DEBUG   ] Finished updating board flags in 2.043 seconds
2018-11-27 00:30:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 00:30:09 [DEBUG   ] Got command BAM from MCS: ref #638204
2018-11-27 00:30:09 [INFO    ] Finished processing BAM request
2018-11-27 00:30:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:30:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638204   8 58449  1809029 A  ERROR'
2018-11-27 00:30:09 [DEBUG   ] Got command BAM from MCS: ref #638203
2018-11-27 00:30:09 [INFO    ] Finished processing BAM request
2018-11-27 00:30:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:30:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638203   8 58449  1809033 A  ERROR'
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:30:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:30:09 [DEBUG   ] spawn_process: at 2018-11-27 00:30:09 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:30:26 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 00:30:46 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 00:30:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:30:48 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:30:48 [DEBUG   ] Got command RPT from MCS: ref #638205
2018-11-27 00:30:48 [DEBUG   ] summary = ERROR
2018-11-27 00:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638205  13 58449  1848849 A  ERRORERROR'
2018-11-27 00:30:48 [DEBUG   ] Got command RPT from MCS: ref #638206
2018-11-27 00:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638206  77 58449  1848850 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:30:49 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:30:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:30:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:30:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:30:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:30:49 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:30:49 [DEBUG   ] Finished updating board flags in 2.015 seconds
2018-11-27 00:30:50 [DEBUG   ] Got command BAM from MCS: ref #638223
2018-11-27 00:30:50 [INFO    ] Finished processing BAM request
2018-11-27 00:30:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:30:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638223   8 58449  1850029 A  ERROR'
2018-11-27 00:30:50 [DEBUG   ] Got command BAM from MCS: ref #638222
2018-11-27 00:30:50 [INFO    ] Finished processing BAM request
2018-11-27 00:30:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:30:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638222   8 58449  1850033 A  ERROR'
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:30:50 [DEBUG   ] spawn_process: at 2018-11-27 00:30:50 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:31:07 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 00:31:27 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 00:31:31 [DEBUG   ] Got command BAM from MCS: ref #638228
2018-11-27 00:31:31 [INFO    ] Finished processing BAM request
2018-11-27 00:31:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:31:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638228   8 58449  1891025 A  ERROR'
2018-11-27 00:31:31 [DEBUG   ] Got command BAM from MCS: ref #638229
2018-11-27 00:31:31 [INFO    ] Finished processing BAM request
2018-11-27 00:31:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:31:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638229   8 58449  1891029 A  ERROR'
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:31:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:31:31 [DEBUG   ] spawn_process: at 2018-11-27 00:31:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:31:47 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 00:31:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 00:31:48 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:31:49 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:31:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:31:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:31:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:31:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:31:49 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:31:49 [DEBUG   ] Finished updating board flags in 1.959 seconds
2018-11-27 00:32:07 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 00:32:12 [DEBUG   ] Got command BAM from MCS: ref #638231
2018-11-27 00:32:12 [INFO    ] Finished processing BAM request
2018-11-27 00:32:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:32:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638231   8 58449  1932033 A  ERROR'
2018-11-27 00:32:12 [DEBUG   ] Got command BAM from MCS: ref #638230
2018-11-27 00:32:12 [INFO    ] Finished processing BAM request
2018-11-27 00:32:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:32:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638230   8 58449  1932037 A  ERROR'
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:32:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:32:12 [DEBUG   ] spawn_process: at 2018-11-27 00:32:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:32:27 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 00:32:47 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 00:32:48 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 00:32:48 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:32:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:32:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:32:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:32:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:32:49 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:32:49 [DEBUG   ] Finished updating board flags in 1.877 seconds
2018-11-27 00:32:52 [DEBUG   ] Got command RPT from MCS: ref #638236
2018-11-27 00:32:52 [DEBUG   ] summary = ERROR
2018-11-27 00:32:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638236  13 58449  1972813 A  ERRORERROR'
2018-11-27 00:32:53 [DEBUG   ] Got command RPT from MCS: ref #638237
2018-11-27 00:32:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:32:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638237  77 58449  1973017 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:32:53 [DEBUG   ] Got command BAM from MCS: ref #638235
2018-11-27 00:32:53 [INFO    ] Finished processing BAM request
2018-11-27 00:32:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:32:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638235   8 58449  1973033 A  ERROR'
2018-11-27 00:32:53 [DEBUG   ] Got command BAM from MCS: ref #638234
2018-11-27 00:32:53 [INFO    ] Finished processing BAM request
2018-11-27 00:32:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:32:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638234   8 58449  1973037 A  ERROR'
2018-11-27 00:32:53 [DEBUG   ] Got command RPT from MCS: ref #638238
2018-11-27 00:32:53 [DEBUG   ] summary = ERROR
2018-11-27 00:32:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638238  13 58449  1973217 A  ERRORERROR'
2018-11-27 00:32:53 [DEBUG   ] Got command RPT from MCS: ref #638240
2018-11-27 00:32:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:32:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638240  77 58449  1973421 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:32:53 [DEBUG   ] Got command RPT from MCS: ref #638241
2018-11-27 00:32:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:32:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638241  12 58449  1973625 A  ERRORBŸff'
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:32:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:32:53 [DEBUG   ] spawn_process: at 2018-11-27 00:32:53 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:33:07 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 00:33:27 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 00:33:34 [DEBUG   ] Got command BAM from MCS: ref #638247
2018-11-27 00:33:34 [INFO    ] Finished processing BAM request
2018-11-27 00:33:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:33:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638247   8 58449  2014029 A  ERROR'
2018-11-27 00:33:34 [DEBUG   ] Got command BAM from MCS: ref #638246
2018-11-27 00:33:34 [INFO    ] Finished processing BAM request
2018-11-27 00:33:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:33:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638246   8 58449  2014033 A  ERROR'
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:33:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:33:34 [DEBUG   ] spawn_process: at 2018-11-27 00:33:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:33:47 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 00:33:48 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:33:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 00:33:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:33:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:33:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:33:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:33:49 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:33:49 [DEBUG   ] Finished updating board flags in 1.732 seconds
2018-11-27 00:34:07 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 00:34:15 [DEBUG   ] Got command BAM from MCS: ref #638252
2018-11-27 00:34:15 [INFO    ] Finished processing BAM request
2018-11-27 00:34:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:34:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638252   8 58449  2055029 A  ERROR'
2018-11-27 00:34:15 [DEBUG   ] Got command BAM from MCS: ref #638253
2018-11-27 00:34:15 [INFO    ] Finished processing BAM request
2018-11-27 00:34:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:34:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638253   8 58449  2055033 A  ERROR'
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:34:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:34:15 [DEBUG   ] spawn_process: at 2018-11-27 00:34:15 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:34:27 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 00:34:47 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-27 00:34:48 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:34:49 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:34:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:34:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:34:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:34:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:34:49 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:34:49 [DEBUG   ] Finished updating board flags in 1.613 seconds
2018-11-27 00:34:56 [DEBUG   ] Got command BAM from MCS: ref #638254
2018-11-27 00:34:56 [INFO    ] Finished processing BAM request
2018-11-27 00:34:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:34:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638254   8 58449  2096033 A  ERROR'
2018-11-27 00:34:56 [DEBUG   ] Got command BAM from MCS: ref #638255
2018-11-27 00:34:56 [INFO    ] Finished processing BAM request
2018-11-27 00:34:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:34:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638255   8 58449  2096037 A  ERROR'
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:34:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:34:56 [DEBUG   ] spawn_process: at 2018-11-27 00:34:56 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:35:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 00:35:28 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-27 00:35:37 [DEBUG   ] Got command BAM from MCS: ref #638257
2018-11-27 00:35:37 [INFO    ] Finished processing BAM request
2018-11-27 00:35:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:35:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638257   8 58449  2137029 A  ERROR'
2018-11-27 00:35:37 [DEBUG   ] Got command BAM from MCS: ref #638256
2018-11-27 00:35:37 [INFO    ] Finished processing BAM request
2018-11-27 00:35:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:35:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638256   8 58449  2137033 A  ERROR'
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:35:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:35:37 [DEBUG   ] spawn_process: at 2018-11-27 00:35:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:35:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:35:48 [DEBUG   ] Finished updating ant stats in 1.251 seconds
2018-11-27 00:35:48 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:35:48 [DEBUG   ] Got command RPT from MCS: ref #638258
2018-11-27 00:35:48 [DEBUG   ] summary = ERROR
2018-11-27 00:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638258  13 58449  2148853 A  ERRORERROR'
2018-11-27 00:35:48 [DEBUG   ] Got command RPT from MCS: ref #638259
2018-11-27 00:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638259  77 58449  2148854 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:35:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 00:35:49 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 00:35:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:35:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:35:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:35:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:35:49 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:35:49 [DEBUG   ] Finished updating board flags in 1.589 seconds
2018-11-27 00:36:08 [DEBUG   ] Finished updating ant stats in 1.156 seconds
2018-11-27 00:36:18 [DEBUG   ] Got command BAM from MCS: ref #638276
2018-11-27 00:36:18 [INFO    ] Finished processing BAM request
2018-11-27 00:36:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:36:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638276   8 58449  2178033 A  ERROR'
2018-11-27 00:36:18 [DEBUG   ] Got command BAM from MCS: ref #638275
2018-11-27 00:36:18 [INFO    ] Finished processing BAM request
2018-11-27 00:36:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:36:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638275   8 58449  2178037 A  ERROR'
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:36:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:36:18 [DEBUG   ] spawn_process: at 2018-11-27 00:36:18 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:36:28 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 00:36:35 [DEBUG   ] Got command STP from MCS: ref #638277
2018-11-27 00:36:35 [INFO    ] Finished processing BAM request
2018-11-27 00:36:35 [INFO    ] Shutdown the adder chain on beam #2
2018-11-27 00:36:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   638277   8 58449  2195025 A  ERROR'
2018-11-27 00:36:35 [DEBUG   ] Got command STP from MCS: ref #638278
2018-11-27 00:36:35 [INFO    ] Finished processing BAM request
2018-11-27 00:36:35 [INFO    ] Shutdown the adder chain on beam #3
2018-11-27 00:36:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   638278   8 58449  2195029 A  ERROR'
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:36:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:36:35 [DEBUG   ] spawn_process: at 2018-11-27 00:36:35 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:36:48 [DEBUG   ] Finished updating ant stats in 0.845 seconds
2018-11-27 00:36:48 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:36:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:36:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:36:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:36:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:36:49 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:36:49 [DEBUG   ] Finished updating board flags in 1.400 seconds
2018-11-27 00:37:08 [DEBUG   ] Finished updating ant stats in 0.757 seconds
2018-11-27 00:37:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 00:37:48 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 00:37:49 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:37:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:37:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:37:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:37:49 [WARNING ] (13, 1, 'bot')
2018-11-27 00:37:49 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:37:49 [DEBUG   ] Finished updating board flags in 1.351 seconds
2018-11-27 00:37:52 [DEBUG   ] Got command RPT from MCS: ref #638283
2018-11-27 00:37:52 [DEBUG   ] summary = ERROR
2018-11-27 00:37:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638283  13 58449  2272909 A  ERRORERROR'
2018-11-27 00:37:53 [DEBUG   ] Got command RPT from MCS: ref #638284
2018-11-27 00:37:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:37:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638284  77 58449  2273101 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:37:53 [DEBUG   ] Got command RPT from MCS: ref #638285
2018-11-27 00:37:53 [DEBUG   ] summary = ERROR
2018-11-27 00:37:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638285  13 58449  2273305 A  ERRORERROR'
2018-11-27 00:37:53 [DEBUG   ] Got command RPT from MCS: ref #638286
2018-11-27 00:37:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:37:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638286  77 58449  2273509 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:37:53 [DEBUG   ] Got command RPT from MCS: ref #638287
2018-11-27 00:37:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:37:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638287  12 58449  2273713 A  ERRORBš™š'
2018-11-27 00:38:09 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 00:38:29 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 00:38:49 [DEBUG   ] Finished updating ant stats in 1.253 seconds
2018-11-27 00:38:49 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:38:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:38:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:38:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:38:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:38:50 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:38:50 [DEBUG   ] Finished updating board flags in 2.065 seconds
2018-11-27 00:39:09 [DEBUG   ] Finished updating ant stats in 1.156 seconds
2018-11-27 00:39:29 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 00:39:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:39:49 [DEBUG   ] Finished updating ant stats in 1.045 seconds
2018-11-27 00:39:49 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:39:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:39:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:39:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:39:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:39:50 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:39:50 [DEBUG   ] Finished updating board flags in 1.986 seconds
2018-11-27 00:40:09 [DEBUG   ] Finished updating ant stats in 0.954 seconds
2018-11-27 00:40:29 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 00:40:48 [DEBUG   ] Got command RPT from MCS: ref #638292
2018-11-27 00:40:48 [DEBUG   ] summary = ERROR
2018-11-27 00:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638292  13 58449  2448858 A  ERRORERROR'
2018-11-27 00:40:48 [DEBUG   ] Got command RPT from MCS: ref #638293
2018-11-27 00:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638293  77 58449  2448858 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:40:49 [DEBUG   ] Finished updating ant stats in 0.847 seconds
2018-11-27 00:40:49 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:40:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:40:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:40:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:40:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:40:50 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:40:50 [DEBUG   ] Finished updating board flags in 1.817 seconds
2018-11-27 00:41:09 [DEBUG   ] Finished updating ant stats in 0.754 seconds
2018-11-27 00:41:29 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 00:41:49 [DEBUG   ] Finished updating ant stats in 0.651 seconds
2018-11-27 00:41:49 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:41:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:41:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:41:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:41:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:41:50 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:41:50 [DEBUG   ] Finished updating board flags in 1.782 seconds
2018-11-27 00:42:09 [DEBUG   ] Finished updating ant stats in 0.557 seconds
2018-11-27 00:42:30 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2018-11-27 00:42:49 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:42:50 [DEBUG   ] Finished updating ant stats in 1.252 seconds
2018-11-27 00:42:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:42:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:42:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:42:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:42:50 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:42:50 [DEBUG   ] Finished updating board flags in 1.675 seconds
2018-11-27 00:42:52 [DEBUG   ] Got command RPT from MCS: ref #638311
2018-11-27 00:42:52 [DEBUG   ] summary = ERROR
2018-11-27 00:42:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638311  13 58449  2572986 A  ERRORERROR'
2018-11-27 00:42:53 [DEBUG   ] Got command RPT from MCS: ref #638312
2018-11-27 00:42:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:42:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638312  77 58449  2573189 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:42:53 [DEBUG   ] Got command RPT from MCS: ref #638313
2018-11-27 00:42:53 [DEBUG   ] summary = ERROR
2018-11-27 00:42:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638313  13 58449  2573394 A  ERRORERROR'
2018-11-27 00:42:53 [DEBUG   ] Got command RPT from MCS: ref #638315
2018-11-27 00:42:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:42:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638315  77 58449  2573597 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:42:53 [DEBUG   ] Got command RPT from MCS: ref #638316
2018-11-27 00:42:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:42:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638316  12 58449  2573790 A  ERRORB–™š'
2018-11-27 00:43:10 [DEBUG   ] Finished updating ant stats in 1.155 seconds
2018-11-27 00:43:30 [DEBUG   ] Got command DRX from MCS: ref #638330
2018-11-27 00:43:30 [INFO    ] sending drx command to lwa16
2018-11-27 00:43:30 [DEBUG   ] run_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 64500000.0 7 9 0']
2018-11-27 00:43:30 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 00:43:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   638330   8 58449  2610014 A  ERROR'
2018-11-27 00:43:30 [DEBUG   ] Got command DRX from MCS: ref #638329
2018-11-27 00:43:30 [INFO    ] sending drx command to lwa16
2018-11-27 00:43:30 [DEBUG   ] run_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 35100000.0 7 9 0']
2018-11-27 00:43:30 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 00:43:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   638329   8 58449  2610018 A  ERROR'
2018-11-27 00:43:30 [DEBUG   ] Got command DRX from MCS: ref #638332
2018-11-27 00:43:30 [INFO    ] sending drx command to lwa16
2018-11-27 00:43:30 [DEBUG   ] run_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 79200000.0 7 9 0']
2018-11-27 00:43:30 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 00:43:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   638332   8 58449  2610022 A  ERROR'
2018-11-27 00:43:30 [DEBUG   ] Got command DRX from MCS: ref #638331
2018-11-27 00:43:30 [INFO    ] sending drx command to lwa16
2018-11-27 00:43:30 [DEBUG   ] run_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 49800000.0 7 9 0']
2018-11-27 00:43:30 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 00:43:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   638331   8 58449  2610026 A  ERROR'
2018-11-27 00:43:30 [DEBUG   ] Got command BAM from MCS: ref #638333
2018-11-27 00:43:30 [INFO    ] Finished processing BAM request
2018-11-27 00:43:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:43:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638333   8 58449  2610029 A  ERROR'
2018-11-27 00:43:30 [DEBUG   ] Got command BAM from MCS: ref #638334
2018-11-27 00:43:30 [INFO    ] Finished processing BAM request
2018-11-27 00:43:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:43:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638334   8 58449  2610033 A  ERROR'
2018-11-27 00:43:30 [INFO    ] Finished starting DRX
2018-11-27 00:43:30 [INFO    ] Finished starting DRX
2018-11-27 00:43:30 [INFO    ] Finished starting DRX
2018-11-27 00:43:30 [INFO    ] Finished starting DRX
2018-11-27 00:43:30 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:43:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:43:30 [DEBUG   ] spawn_process: at 2018-11-27 00:43:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:43:49 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:43:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 00:43:50 [DEBUG   ] Finished updating ant stats in 1.045 seconds
2018-11-27 00:43:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:43:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:43:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:43:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:43:50 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:43:50 [DEBUG   ] Finished updating board flags in 1.585 seconds
2018-11-27 00:44:10 [DEBUG   ] Finished updating ant stats in 0.961 seconds
2018-11-27 00:44:30 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 00:44:33 [DEBUG   ] Got command BAM from MCS: ref #638338
2018-11-27 00:44:33 [INFO    ] Finished processing BAM request
2018-11-27 00:44:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:44:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638338   8 58449  2673029 A  ERROR'
2018-11-27 00:44:33 [DEBUG   ] Got command BAM from MCS: ref #638337
2018-11-27 00:44:33 [INFO    ] Finished processing BAM request
2018-11-27 00:44:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:44:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638337   8 58449  2673033 A  ERROR'
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:44:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:44:33 [DEBUG   ] spawn_process: at 2018-11-27 00:44:33 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:44:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:44:49 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:44:50 [DEBUG   ] Finished updating ant stats in 0.643 seconds
2018-11-27 00:44:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:44:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:44:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:44:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:44:50 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:44:50 [DEBUG   ] Finished updating board flags in 1.452 seconds
2018-11-27 00:45:10 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2018-11-27 00:45:31 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 00:45:36 [DEBUG   ] Got command BAM from MCS: ref #638340
2018-11-27 00:45:36 [INFO    ] Finished processing BAM request
2018-11-27 00:45:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:45:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638340   8 58449  2736029 A  ERROR'
2018-11-27 00:45:36 [DEBUG   ] Got command BAM from MCS: ref #638339
2018-11-27 00:45:36 [INFO    ] Finished processing BAM request
2018-11-27 00:45:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:45:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638339   8 58449  2736033 A  ERROR'
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:45:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:45:36 [DEBUG   ] spawn_process: at 2018-11-27 00:45:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:45:48 [DEBUG   ] Got command RPT from MCS: ref #638341
2018-11-27 00:45:48 [DEBUG   ] summary = ERROR
2018-11-27 00:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638341  13 58449  2748861 A  ERRORERROR'
2018-11-27 00:45:48 [DEBUG   ] Got command RPT from MCS: ref #638342
2018-11-27 00:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638342  77 58449  2748862 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:45:49 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:45:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:45:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:45:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:45:50 [WARNING ] (13, 1, 'bot')
2018-11-27 00:45:50 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:45:50 [DEBUG   ] Finished updating board flags in 1.393 seconds
2018-11-27 00:45:51 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 00:46:11 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 00:46:31 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 00:46:39 [DEBUG   ] Got command BAM from MCS: ref #638363
2018-11-27 00:46:39 [INFO    ] Finished processing BAM request
2018-11-27 00:46:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:46:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638363   8 58449  2799033 A  ERROR'
2018-11-27 00:46:39 [DEBUG   ] Got command BAM from MCS: ref #638362
2018-11-27 00:46:39 [INFO    ] Finished processing BAM request
2018-11-27 00:46:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:46:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638362   8 58449  2799037 A  ERROR'
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:46:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:46:39 [DEBUG   ] spawn_process: at 2018-11-27 00:46:39 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:46:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 00:46:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:46:51 [DEBUG   ] Finished updating ant stats in 1.242 seconds
2018-11-27 00:46:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 00:46:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:46:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:46:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:46:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:46:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:46:51 [DEBUG   ] Finished updating board flags in 2.166 seconds
2018-11-27 00:47:11 [DEBUG   ] Finished updating ant stats in 1.158 seconds
2018-11-27 00:47:31 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 00:47:42 [DEBUG   ] Got command BAM from MCS: ref #638364
2018-11-27 00:47:42 [INFO    ] Finished processing BAM request
2018-11-27 00:47:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:47:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638364   8 58449  2862033 A  ERROR'
2018-11-27 00:47:42 [DEBUG   ] Got command BAM from MCS: ref #638365
2018-11-27 00:47:42 [INFO    ] Finished processing BAM request
2018-11-27 00:47:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:47:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638365   8 58449  2862037 A  ERROR'
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:47:42 [DEBUG   ] spawn_process: at 2018-11-27 00:47:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:47:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:47:51 [DEBUG   ] Finished updating ant stats in 1.048 seconds
2018-11-27 00:47:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:47:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:47:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:47:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:47:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:47:51 [DEBUG   ] Finished updating board flags in 2.002 seconds
2018-11-27 00:47:53 [DEBUG   ] Got command RPT from MCS: ref #638366
2018-11-27 00:47:53 [DEBUG   ] summary = ERROR
2018-11-27 00:47:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638366  13 58449  2873081 A  ERRORERROR'
2018-11-27 00:47:53 [DEBUG   ] Got command RPT from MCS: ref #638367
2018-11-27 00:47:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:47:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638367  77 58449  2873273 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:47:53 [DEBUG   ] Got command RPT from MCS: ref #638368
2018-11-27 00:47:53 [DEBUG   ] summary = ERROR
2018-11-27 00:47:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638368  13 58449  2873477 A  ERRORERROR'
2018-11-27 00:47:53 [DEBUG   ] Got command RPT from MCS: ref #638369
2018-11-27 00:47:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:47:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638369  77 58449  2873681 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:47:53 [DEBUG   ] Got command RPT from MCS: ref #638370
2018-11-27 00:47:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:47:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638370  12 58449  2873885 A  ERRORBžff'
2018-11-27 00:48:11 [DEBUG   ] Finished updating ant stats in 0.956 seconds
2018-11-27 00:48:31 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 00:48:45 [DEBUG   ] Got command BAM from MCS: ref #638372
2018-11-27 00:48:45 [INFO    ] Finished processing BAM request
2018-11-27 00:48:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:48:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638372   8 58449  2925025 A  ERROR'
2018-11-27 00:48:45 [DEBUG   ] Got command BAM from MCS: ref #638371
2018-11-27 00:48:45 [INFO    ] Finished processing BAM request
2018-11-27 00:48:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:48:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638371   8 58449  2925029 A  ERROR'
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:48:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:48:45 [DEBUG   ] spawn_process: at 2018-11-27 00:48:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:48:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:48:51 [DEBUG   ] Finished updating ant stats in 0.643 seconds
2018-11-27 00:48:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:48:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:48:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:48:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:48:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:48:51 [DEBUG   ] Finished updating board flags in 1.998 seconds
2018-11-27 00:49:11 [DEBUG   ] Finished updating ant stats in 0.560 seconds
2018-11-27 00:49:32 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2018-11-27 00:49:48 [DEBUG   ] Got command BAM from MCS: ref #638377
2018-11-27 00:49:48 [INFO    ] Finished processing BAM request
2018-11-27 00:49:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:49:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638377   8 58449  2988033 A  ERROR'
2018-11-27 00:49:48 [DEBUG   ] Got command BAM from MCS: ref #638378
2018-11-27 00:49:48 [INFO    ] Finished processing BAM request
2018-11-27 00:49:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:49:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638378   8 58449  2988037 A  ERROR'
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:49:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:49:48 [DEBUG   ] spawn_process: at 2018-11-27 00:49:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:49:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 00:49:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:49:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 00:49:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:49:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:49:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:49:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:49:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:49:51 [DEBUG   ] Finished updating board flags in 1.790 seconds
2018-11-27 00:49:52 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 00:50:12 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 00:50:32 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 00:50:48 [DEBUG   ] Got command RPT from MCS: ref #638379
2018-11-27 00:50:48 [DEBUG   ] summary = ERROR
2018-11-27 00:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638379  13 58449  3048853 A  ERRORERROR'
2018-11-27 00:50:48 [DEBUG   ] Got command RPT from MCS: ref #638380
2018-11-27 00:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638380  77 58449  3048854 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:50:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:50:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:50:51 [DEBUG   ] Got command BAM from MCS: ref #638396
2018-11-27 00:50:51 [INFO    ] Finished processing BAM request
2018-11-27 00:50:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:50:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638396   8 58449  3051033 A  ERROR'
2018-11-27 00:50:51 [DEBUG   ] Got command BAM from MCS: ref #638397
2018-11-27 00:50:51 [INFO    ] Finished processing BAM request
2018-11-27 00:50:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:50:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638397   8 58449  3051041 A  ERROR'
2018-11-27 00:50:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:50:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:50:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:50:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:50:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:50:51 [DEBUG   ] Finished updating board flags in 1.616 seconds
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:50:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:50:51 [DEBUG   ] spawn_process: at 2018-11-27 00:50:51 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:50:52 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 00:51:12 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 00:51:32 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-27 00:51:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:51:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 00:51:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:51:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:51:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:51:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:51:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:51:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:51:51 [DEBUG   ] Finished updating board flags in 1.598 seconds
2018-11-27 00:51:52 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-27 00:51:53 [DEBUG   ] Got command BAM from MCS: ref #638399
2018-11-27 00:51:53 [INFO    ] Finished processing BAM request
2018-11-27 00:51:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:51:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638399   8 58449  3113025 A  ERROR'
2018-11-27 00:51:53 [DEBUG   ] Got command BAM from MCS: ref #638398
2018-11-27 00:51:53 [INFO    ] Finished processing BAM request
2018-11-27 00:51:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:51:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638398   8 58449  3113029 A  ERROR'
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 00:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 00:51:53 [DEBUG   ] spawn_process: at 2018-11-27 00:51:53 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 00:52:12 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 00:52:32 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 00:52:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:52:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 00:52:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:52:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:52:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:52:51 [WARNING ] (13, 1, 'bot')
2018-11-27 00:52:51 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:52:51 [DEBUG   ] Finished updating board flags in 1.391 seconds
2018-11-27 00:52:52 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-27 00:52:53 [DEBUG   ] Got command RPT from MCS: ref #638406
2018-11-27 00:52:53 [DEBUG   ] summary = ERROR
2018-11-27 00:52:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638406  13 58449  3173157 A  ERRORERROR'
2018-11-27 00:52:53 [DEBUG   ] Got command RPT from MCS: ref #638407
2018-11-27 00:52:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:52:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638407  77 58449  3173361 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:52:53 [DEBUG   ] Got command RPT from MCS: ref #638408
2018-11-27 00:52:53 [DEBUG   ] summary = ERROR
2018-11-27 00:52:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638408  13 58449  3173565 A  ERRORERROR'
2018-11-27 00:52:53 [DEBUG   ] Got command RPT from MCS: ref #638410
2018-11-27 00:52:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:52:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638410  77 58449  3173769 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:52:53 [DEBUG   ] Got command RPT from MCS: ref #638412
2018-11-27 00:52:53 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:52:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638412  12 58449  3173961 A  ERRORBžff'
2018-11-27 00:52:55 [DEBUG   ] Got command BAM from MCS: ref #638414
2018-11-27 00:52:55 [INFO    ] Finished processing BAM request
2018-11-27 00:52:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:52:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638414   8 58449  3175033 A  ERROR'
2018-11-27 00:52:55 [DEBUG   ] Got command BAM from MCS: ref #638413
2018-11-27 00:52:55 [INFO    ] Finished processing BAM request
2018-11-27 00:52:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:52:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638413   8 58449  3175037 A  ERROR'
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 00:52:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 00:52:55 [DEBUG   ] spawn_process: at 2018-11-27 00:52:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 00:53:12 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 00:53:33 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 00:53:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:53:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 00:53:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:53:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:53:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:53:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:53:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:53:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:53:52 [DEBUG   ] Finished updating board flags in 2.210 seconds
2018-11-27 00:53:53 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 00:53:57 [DEBUG   ] Got command BAM from MCS: ref #638418
2018-11-27 00:53:57 [INFO    ] Finished processing BAM request
2018-11-27 00:53:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:53:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638418   8 58449  3237029 A  ERROR'
2018-11-27 00:53:57 [DEBUG   ] Got command BAM from MCS: ref #638419
2018-11-27 00:53:57 [INFO    ] Finished processing BAM request
2018-11-27 00:53:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:53:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638419   8 58449  3237033 A  ERROR'
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:53:57 [DEBUG   ] spawn_process: at 2018-11-27 00:53:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:54:13 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 00:54:33 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 00:54:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:54:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:54:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:54:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:54:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:54:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:54:52 [DEBUG   ] Finished updating board flags in 2.202 seconds
2018-11-27 00:54:53 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-27 00:54:59 [DEBUG   ] Got command BAM from MCS: ref #638420
2018-11-27 00:54:59 [INFO    ] Finished processing BAM request
2018-11-27 00:54:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:54:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638420   8 58449  3299029 A  ERROR'
2018-11-27 00:54:59 [DEBUG   ] Got command BAM from MCS: ref #638421
2018-11-27 00:54:59 [INFO    ] Finished processing BAM request
2018-11-27 00:54:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:54:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638421   8 58449  3299033 A  ERROR'
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:54:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:54:59 [DEBUG   ] spawn_process: at 2018-11-27 00:54:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:55:13 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 00:55:33 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 00:55:48 [DEBUG   ] Got command RPT from MCS: ref #638426
2018-11-27 00:55:48 [DEBUG   ] summary = ERROR
2018-11-27 00:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638426  13 58449  3348861 A  ERRORERROR'
2018-11-27 00:55:48 [DEBUG   ] Got command RPT from MCS: ref #638427
2018-11-27 00:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638427  77 58449  3348862 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:55:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 00:55:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:55:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:55:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:55:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:55:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:55:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:55:52 [DEBUG   ] Finished updating board flags in 2.167 seconds
2018-11-27 00:55:53 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 00:56:01 [DEBUG   ] Got command BAM from MCS: ref #638443
2018-11-27 00:56:01 [INFO    ] Finished processing BAM request
2018-11-27 00:56:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:56:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638443   8 58449  3361025 A  ERROR'
2018-11-27 00:56:01 [DEBUG   ] Got command BAM from MCS: ref #638444
2018-11-27 00:56:01 [INFO    ] Finished processing BAM request
2018-11-27 00:56:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:56:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638444   8 58449  3361029 A  ERROR'
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:56:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:56:01 [DEBUG   ] spawn_process: at 2018-11-27 00:56:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:56:13 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 00:56:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 00:56:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 00:56:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:56:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:56:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:56:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:56:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:56:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:56:52 [DEBUG   ] Finished updating board flags in 2.015 seconds
2018-11-27 00:56:53 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 00:57:03 [DEBUG   ] Got command BAM from MCS: ref #638446
2018-11-27 00:57:03 [INFO    ] Finished processing BAM request
2018-11-27 00:57:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:57:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638446   8 58449  3423029 A  ERROR'
2018-11-27 00:57:03 [DEBUG   ] Got command BAM from MCS: ref #638445
2018-11-27 00:57:03 [INFO    ] Finished processing BAM request
2018-11-27 00:57:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:57:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638445   8 58449  3423033 A  ERROR'
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 00:57:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 00:57:03 [DEBUG   ] spawn_process: at 2018-11-27 00:57:03 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 00:57:13 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 00:57:34 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 00:57:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:57:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 00:57:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:57:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:57:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:57:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:57:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:57:52 [DEBUG   ] Finished updating board flags in 1.985 seconds
2018-11-27 00:57:53 [DEBUG   ] Got command RPT from MCS: ref #638447
2018-11-27 00:57:53 [DEBUG   ] summary = ERROR
2018-11-27 00:57:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638447  13 58449  3473250 A  ERRORERROR'
2018-11-27 00:57:53 [DEBUG   ] Got command RPT from MCS: ref #638448
2018-11-27 00:57:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:57:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638448  77 58449  3473441 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:57:53 [DEBUG   ] Got command RPT from MCS: ref #638449
2018-11-27 00:57:53 [DEBUG   ] summary = ERROR
2018-11-27 00:57:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638449  13 58449  3473645 A  ERRORERROR'
2018-11-27 00:57:53 [DEBUG   ] Got command RPT from MCS: ref #638450
2018-11-27 00:57:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 00:57:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638450  77 58449  3473849 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 00:57:54 [DEBUG   ] Got command RPT from MCS: ref #638451
2018-11-27 00:57:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 00:57:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638451  12 58449  3474053 A  ERRORBžff'
2018-11-27 00:57:54 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-27 00:58:05 [DEBUG   ] Got command BAM from MCS: ref #638456
2018-11-27 00:58:05 [INFO    ] Finished processing BAM request
2018-11-27 00:58:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:58:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638456   8 58449  3485025 A  ERROR'
2018-11-27 00:58:05 [DEBUG   ] Got command BAM from MCS: ref #638455
2018-11-27 00:58:05 [INFO    ] Finished processing BAM request
2018-11-27 00:58:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:58:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638455   8 58449  3485029 A  ERROR'
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 00:58:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 00:58:05 [DEBUG   ] spawn_process: at 2018-11-27 00:58:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 00:58:14 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 00:58:34 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 00:58:51 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 00:58:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:58:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:58:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:58:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:58:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:58:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:58:52 [DEBUG   ] Finished updating board flags in 1.811 seconds
2018-11-27 00:58:54 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 00:59:07 [DEBUG   ] Got command BAM from MCS: ref #638458
2018-11-27 00:59:07 [INFO    ] Finished processing BAM request
2018-11-27 00:59:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:59:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638458   8 58449  3547025 A  ERROR'
2018-11-27 00:59:07 [DEBUG   ] Got command BAM from MCS: ref #638459
2018-11-27 00:59:07 [INFO    ] Finished processing BAM request
2018-11-27 00:59:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 00:59:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638459   8 58449  3547029 A  ERROR'
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 00:59:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 00:59:07 [DEBUG   ] spawn_process: at 2018-11-27 00:59:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 00:59:14 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 00:59:34 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 00:59:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 00:59:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:59:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:59:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:59:52 [WARNING ] (13, 1, 'bot')
2018-11-27 00:59:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 00:59:52 [DEBUG   ] Finished updating board flags in 1.794 seconds
2018-11-27 00:59:54 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2018-11-27 01:00:09 [DEBUG   ] Got command BAM from MCS: ref #638461
2018-11-27 01:00:09 [INFO    ] Finished processing BAM request
2018-11-27 01:00:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:00:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638461   8 58449  3609025 A  ERROR'
2018-11-27 01:00:09 [DEBUG   ] Got command BAM from MCS: ref #638460
2018-11-27 01:00:09 [INFO    ] Finished processing BAM request
2018-11-27 01:00:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:00:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638460   8 58449  3609029 A  ERROR'
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 01:00:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 01:00:09 [DEBUG   ] spawn_process: at 2018-11-27 01:00:09 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 01:00:14 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 01:00:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:00:48 [DEBUG   ] Got command RPT from MCS: ref #638462
2018-11-27 01:00:48 [DEBUG   ] summary = ERROR
2018-11-27 01:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638462  13 58449  3648861 A  ERRORERROR'
2018-11-27 01:00:48 [DEBUG   ] Got command RPT from MCS: ref #638463
2018-11-27 01:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638463  77 58449  3648862 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:00:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:00:52 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 01:00:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 01:00:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:00:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:00:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:00:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:00:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:00:52 [DEBUG   ] Finished updating board flags in 1.600 seconds
2018-11-27 01:00:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 01:01:11 [DEBUG   ] Got command BAM from MCS: ref #638484
2018-11-27 01:01:11 [INFO    ] Finished processing BAM request
2018-11-27 01:01:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:01:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638484   8 58449  3671033 A  ERROR'
2018-11-27 01:01:11 [DEBUG   ] Got command BAM from MCS: ref #638483
2018-11-27 01:01:11 [INFO    ] Finished processing BAM request
2018-11-27 01:01:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:01:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638483   8 58449  3671037 A  ERROR'
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:01:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:01:11 [DEBUG   ] spawn_process: at 2018-11-27 01:01:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:01:14 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-27 01:01:34 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 01:01:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:01:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 01:01:52 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 01:01:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:01:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:01:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:01:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:01:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:01:52 [DEBUG   ] Finished updating board flags in 1.633 seconds
2018-11-27 01:01:54 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 01:02:13 [DEBUG   ] Got command BAM from MCS: ref #638485
2018-11-27 01:02:13 [INFO    ] Finished processing BAM request
2018-11-27 01:02:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:02:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638485   8 58449  3733025 A  ERROR'
2018-11-27 01:02:13 [DEBUG   ] Got command BAM from MCS: ref #638486
2018-11-27 01:02:13 [INFO    ] Finished processing BAM request
2018-11-27 01:02:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:02:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638486   8 58449  3733029 A  ERROR'
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 01:02:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 01:02:13 [DEBUG   ] spawn_process: at 2018-11-27 01:02:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 01:02:14 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 01:02:34 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-27 01:02:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:02:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:02:52 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:02:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:02:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:02:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:02:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:02:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:02:52 [DEBUG   ] Finished updating board flags in 1.599 seconds
2018-11-27 01:02:53 [DEBUG   ] Got command RPT from MCS: ref #638489
2018-11-27 01:02:53 [DEBUG   ] summary = ERROR
2018-11-27 01:02:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638489  13 58449  3773329 A  ERRORERROR'
2018-11-27 01:02:53 [DEBUG   ] Got command RPT from MCS: ref #638490
2018-11-27 01:02:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:02:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638490  77 58449  3773533 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:02:53 [DEBUG   ] Got command RPT from MCS: ref #638491
2018-11-27 01:02:53 [DEBUG   ] summary = ERROR
2018-11-27 01:02:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638491  13 58449  3773738 A  ERRORERROR'
2018-11-27 01:02:53 [DEBUG   ] Got command RPT from MCS: ref #638492
2018-11-27 01:02:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:02:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638492  77 58449  3773929 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:02:54 [DEBUG   ] Got command RPT from MCS: ref #638494
2018-11-27 01:02:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:02:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638494  12 58449  3774133 A  ERRORB¡ff'
2018-11-27 01:02:54 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 01:03:14 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 01:03:15 [DEBUG   ] Got command BAM from MCS: ref #638499
2018-11-27 01:03:15 [INFO    ] Finished processing BAM request
2018-11-27 01:03:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:03:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638499   8 58449  3795029 A  ERROR'
2018-11-27 01:03:15 [DEBUG   ] Got command BAM from MCS: ref #638500
2018-11-27 01:03:15 [INFO    ] Finished processing BAM request
2018-11-27 01:03:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:03:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638500   8 58449  3795033 A  ERROR'
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 01:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 01:03:15 [DEBUG   ] spawn_process: at 2018-11-27 01:03:15 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 01:03:34 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 01:03:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:03:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 01:03:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:03:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 01:03:52 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:03:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 01:03:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:03:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:03:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:03:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:03:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:03:52 [DEBUG   ] Finished updating board flags in 1.366 seconds
2018-11-27 01:03:55 [DEBUG   ] Finished updating ant stats in 1.385 seconds
2018-11-27 01:04:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:04:17 [DEBUG   ] Got command BAM from MCS: ref #638506
2018-11-27 01:04:17 [INFO    ] Finished processing BAM request
2018-11-27 01:04:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:04:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638506   8 58449  3857029 A  ERROR'
2018-11-27 01:04:17 [DEBUG   ] Got command BAM from MCS: ref #638505
2018-11-27 01:04:17 [INFO    ] Finished processing BAM request
2018-11-27 01:04:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:04:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638505   8 58449  3857033 A  ERROR'
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:04:17 [DEBUG   ] spawn_process: at 2018-11-27 01:04:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:04:35 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 01:04:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:04:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:04:52 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:04:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:04:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:04:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:04:52 [WARNING ] (13, 1, 'bot')
2018-11-27 01:04:52 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:04:52 [DEBUG   ] Finished updating board flags in 1.367 seconds
2018-11-27 01:04:55 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:05:15 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2018-11-27 01:05:19 [DEBUG   ] Got command BAM from MCS: ref #638507
2018-11-27 01:05:19 [INFO    ] Finished processing BAM request
2018-11-27 01:05:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:05:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638507   8 58449  3919029 A  ERROR'
2018-11-27 01:05:19 [DEBUG   ] Got command BAM from MCS: ref #638508
2018-11-27 01:05:19 [INFO    ] Finished processing BAM request
2018-11-27 01:05:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:05:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638508   8 58449  3919033 A  ERROR'
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:05:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:05:19 [DEBUG   ] spawn_process: at 2018-11-27 01:05:19 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:05:35 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:05:48 [DEBUG   ] Got command RPT from MCS: ref #638509
2018-11-27 01:05:48 [DEBUG   ] summary = ERROR
2018-11-27 01:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638509  13 58449  3948865 A  ERRORERROR'
2018-11-27 01:05:48 [DEBUG   ] Got command RPT from MCS: ref #638510
2018-11-27 01:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638510  77 58449  3948866 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:05:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:05:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:05:53 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:05:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:05:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:05:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:05:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:05:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:05:53 [DEBUG   ] Finished updating board flags in 2.049 seconds
2018-11-27 01:05:55 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 01:06:15 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-27 01:06:21 [DEBUG   ] Got command BAM from MCS: ref #638526
2018-11-27 01:06:21 [INFO    ] Finished processing BAM request
2018-11-27 01:06:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:06:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638526   8 58449  3981025 A  ERROR'
2018-11-27 01:06:21 [DEBUG   ] Got command BAM from MCS: ref #638527
2018-11-27 01:06:21 [INFO    ] Finished processing BAM request
2018-11-27 01:06:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:06:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638527   8 58449  3981029 A  ERROR'
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:06:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:06:21 [DEBUG   ] spawn_process: at 2018-11-27 01:06:21 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:06:35 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 01:06:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:06:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:06:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 01:06:53 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:06:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:06:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:06:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:06:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:06:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:06:53 [DEBUG   ] Finished updating board flags in 2.021 seconds
2018-11-27 01:06:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:07:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 01:07:23 [DEBUG   ] Got command BAM from MCS: ref #638532
2018-11-27 01:07:23 [INFO    ] Finished processing BAM request
2018-11-27 01:07:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:07:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638532   8 58449  4043033 A  ERROR'
2018-11-27 01:07:23 [DEBUG   ] Got command BAM from MCS: ref #638533
2018-11-27 01:07:23 [INFO    ] Finished processing BAM request
2018-11-27 01:07:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:07:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638533   8 58449  4043037 A  ERROR'
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 01:07:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 01:07:23 [DEBUG   ] spawn_process: at 2018-11-27 01:07:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 01:07:35 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 01:07:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:07:53 [DEBUG   ] Got command RPT from MCS: ref #638534
2018-11-27 01:07:53 [DEBUG   ] summary = ERROR
2018-11-27 01:07:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638534  13 58449  4073421 A  ERRORERROR'
2018-11-27 01:07:53 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:07:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 01:07:53 [DEBUG   ] Got command RPT from MCS: ref #638535
2018-11-27 01:07:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:07:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638535  77 58449  4073613 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:07:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:07:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:07:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:07:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:07:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:07:53 [DEBUG   ] Finished updating board flags in 2.011 seconds
2018-11-27 01:07:53 [DEBUG   ] Got command RPT from MCS: ref #638536
2018-11-27 01:07:53 [DEBUG   ] summary = ERROR
2018-11-27 01:07:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638536  13 58449  4073817 A  ERRORERROR'
2018-11-27 01:07:54 [DEBUG   ] Got command RPT from MCS: ref #638537
2018-11-27 01:07:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:07:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638537  77 58449  4074029 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:07:54 [DEBUG   ] Got command RPT from MCS: ref #638538
2018-11-27 01:07:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:07:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638538  12 58449  4074217 A  ERRORB¡ff'
2018-11-27 01:07:55 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 01:08:16 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2018-11-27 01:08:25 [DEBUG   ] Got command BAM from MCS: ref #638540
2018-11-27 01:08:25 [INFO    ] Finished processing BAM request
2018-11-27 01:08:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:08:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638540   8 58449  4105025 A  ERROR'
2018-11-27 01:08:25 [DEBUG   ] Got command BAM from MCS: ref #638539
2018-11-27 01:08:25 [INFO    ] Finished processing BAM request
2018-11-27 01:08:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:08:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638539   8 58449  4105029 A  ERROR'
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 01:08:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 01:08:25 [DEBUG   ] spawn_process: at 2018-11-27 01:08:25 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 01:08:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 01:08:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:08:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:08:53 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:08:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:08:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:08:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:08:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:08:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:08:53 [DEBUG   ] Finished updating board flags in 1.969 seconds
2018-11-27 01:08:55 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-27 01:09:15 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2018-11-27 01:09:27 [DEBUG   ] Got command BAM from MCS: ref #638542
2018-11-27 01:09:27 [INFO    ] Finished processing BAM request
2018-11-27 01:09:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:09:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638542   8 58449  4167025 A  ERROR'
2018-11-27 01:09:27 [DEBUG   ] Got command BAM from MCS: ref #638541
2018-11-27 01:09:27 [INFO    ] Finished processing BAM request
2018-11-27 01:09:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:09:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638541   8 58449  4167029 A  ERROR'
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 01:09:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 01:09:27 [DEBUG   ] spawn_process: at 2018-11-27 01:09:27 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 01:09:35 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 01:09:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:09:53 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:09:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:09:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:09:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:09:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:09:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:09:53 [DEBUG   ] Finished updating board flags in 1.804 seconds
2018-11-27 01:09:56 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2018-11-27 01:10:15 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-27 01:10:29 [DEBUG   ] Got command BAM from MCS: ref #638548
2018-11-27 01:10:29 [INFO    ] Finished processing BAM request
2018-11-27 01:10:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:10:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638548   8 58449  4229029 A  ERROR'
2018-11-27 01:10:29 [DEBUG   ] Got command BAM from MCS: ref #638547
2018-11-27 01:10:29 [INFO    ] Finished processing BAM request
2018-11-27 01:10:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:10:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638547   8 58449  4229033 A  ERROR'
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:10:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:10:29 [DEBUG   ] spawn_process: at 2018-11-27 01:10:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:10:36 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2018-11-27 01:10:48 [DEBUG   ] Got command RPT from MCS: ref #638549
2018-11-27 01:10:48 [DEBUG   ] summary = ERROR
2018-11-27 01:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638549  13 58449  4248861 A  ERRORERROR'
2018-11-27 01:10:48 [DEBUG   ] Got command RPT from MCS: ref #638550
2018-11-27 01:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638550  77 58449  4248862 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:10:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:10:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:10:53 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 01:10:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:10:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:10:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:10:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:10:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:10:53 [DEBUG   ] Finished updating board flags in 1.795 seconds
2018-11-27 01:10:56 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-27 01:11:16 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 01:11:30 [DEBUG   ] Got command BAM from MCS: ref #638566
2018-11-27 01:11:30 [INFO    ] Finished processing BAM request
2018-11-27 01:11:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:11:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638566   8 58449  4290029 A  ERROR'
2018-11-27 01:11:30 [DEBUG   ] Got command BAM from MCS: ref #638567
2018-11-27 01:11:30 [INFO    ] Finished processing BAM request
2018-11-27 01:11:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:11:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638567   8 58449  4290033 A  ERROR'
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:11:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:11:30 [DEBUG   ] spawn_process: at 2018-11-27 01:11:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:11:36 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 01:11:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 01:11:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:11:53 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:11:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:11:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:11:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:11:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:11:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:11:53 [DEBUG   ] Finished updating board flags in 1.611 seconds
2018-11-27 01:11:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:12:16 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:12:31 [DEBUG   ] Got command BAM from MCS: ref #638569
2018-11-27 01:12:31 [INFO    ] Finished processing BAM request
2018-11-27 01:12:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:12:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638569   8 58449  4351029 A  ERROR'
2018-11-27 01:12:31 [DEBUG   ] Got command BAM from MCS: ref #638568
2018-11-27 01:12:31 [INFO    ] Finished processing BAM request
2018-11-27 01:12:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:12:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638568   8 58449  4351033 A  ERROR'
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:12:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:12:31 [DEBUG   ] spawn_process: at 2018-11-27 01:12:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:12:36 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 01:12:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:12:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2018-11-27 01:12:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 5
2018-11-27 01:12:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:12:53 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:12:53 [DEBUG   ] Got command RPT from MCS: ref #638572
2018-11-27 01:12:53 [DEBUG   ] summary = ERROR
2018-11-27 01:12:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638572  13 58449  4373505 A  ERRORERROR'
2018-11-27 01:12:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:12:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:12:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:12:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:12:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:12:53 [DEBUG   ] Finished updating board flags in 1.589 seconds
2018-11-27 01:12:53 [DEBUG   ] Got command RPT from MCS: ref #638573
2018-11-27 01:12:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:12:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638573  77 58449  4373709 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:12:53 [DEBUG   ] Got command RPT from MCS: ref #638574
2018-11-27 01:12:53 [DEBUG   ] summary = ERROR
2018-11-27 01:12:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638574  13 58449  4373901 A  ERRORERROR'
2018-11-27 01:12:54 [DEBUG   ] Got command RPT from MCS: ref #638576
2018-11-27 01:12:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:12:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638576  77 58449  4374105 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:12:54 [DEBUG   ] Got command RPT from MCS: ref #638577
2018-11-27 01:12:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:12:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638577  12 58449  4374309 A  ERRORB¡ff'
2018-11-27 01:12:56 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 01:13:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:13:32 [DEBUG   ] Got command BAM from MCS: ref #638586
2018-11-27 01:13:32 [INFO    ] Finished processing BAM request
2018-11-27 01:13:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:13:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638586   8 58449  4412025 A  ERROR'
2018-11-27 01:13:32 [DEBUG   ] Got command BAM from MCS: ref #638587
2018-11-27 01:13:32 [INFO    ] Finished processing BAM request
2018-11-27 01:13:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:13:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638587   8 58449  4412029 A  ERROR'
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:13:32 [DEBUG   ] spawn_process: at 2018-11-27 01:13:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:13:36 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 01:13:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:13:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:13:53 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:13:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:13:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:13:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:13:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:13:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:13:53 [DEBUG   ] Finished updating board flags in 1.412 seconds
2018-11-27 01:13:56 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 01:14:16 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 01:14:33 [DEBUG   ] Got command BAM from MCS: ref #638589
2018-11-27 01:14:33 [INFO    ] Finished processing BAM request
2018-11-27 01:14:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:14:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638589   8 58449  4473033 A  ERROR'
2018-11-27 01:14:33 [DEBUG   ] Got command BAM from MCS: ref #638588
2018-11-27 01:14:33 [INFO    ] Finished processing BAM request
2018-11-27 01:14:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:14:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638588   8 58449  4473037 A  ERROR'
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:14:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:14:33 [DEBUG   ] spawn_process: at 2018-11-27 01:14:33 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:14:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 01:14:52 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:14:53 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 01:14:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:14:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:14:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:14:53 [WARNING ] (13, 1, 'bot')
2018-11-27 01:14:53 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:14:53 [DEBUG   ] Finished updating board flags in 1.397 seconds
2018-11-27 01:14:56 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 01:15:16 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 01:15:34 [DEBUG   ] Got command BAM from MCS: ref #638590
2018-11-27 01:15:34 [INFO    ] Finished processing BAM request
2018-11-27 01:15:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:15:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638590   8 58449  4534029 A  ERROR'
2018-11-27 01:15:34 [DEBUG   ] Got command BAM from MCS: ref #638591
2018-11-27 01:15:34 [INFO    ] Finished processing BAM request
2018-11-27 01:15:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:15:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638591   8 58449  4534033 A  ERROR'
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 01:15:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 01:15:34 [DEBUG   ] spawn_process: at 2018-11-27 01:15:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 01:15:36 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 01:15:48 [DEBUG   ] Got command RPT from MCS: ref #638592
2018-11-27 01:15:48 [DEBUG   ] summary = ERROR
2018-11-27 01:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638592  13 58449  4548865 A  ERRORERROR'
2018-11-27 01:15:48 [DEBUG   ] Got command RPT from MCS: ref #638593
2018-11-27 01:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638593  77 58449  4548866 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:15:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:15:54 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 01:15:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:15:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:15:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:15:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:15:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:15:54 [DEBUG   ] Finished updating board flags in 2.231 seconds
2018-11-27 01:15:57 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 01:16:17 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-27 01:16:35 [DEBUG   ] Got command BAM from MCS: ref #638613
2018-11-27 01:16:35 [INFO    ] Finished processing BAM request
2018-11-27 01:16:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:16:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638613   8 58449  4595033 A  ERROR'
2018-11-27 01:16:35 [DEBUG   ] Got command BAM from MCS: ref #638614
2018-11-27 01:16:35 [INFO    ] Finished processing BAM request
2018-11-27 01:16:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:16:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638614   8 58449  4595037 A  ERROR'
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:16:35 [DEBUG   ] spawn_process: at 2018-11-27 01:16:35 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:16:37 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 01:16:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:16:54 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 01:16:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:16:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:16:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:16:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:16:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:16:54 [DEBUG   ] Finished updating board flags in 2.163 seconds
2018-11-27 01:16:57 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 01:17:17 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 01:17:37 [DEBUG   ] Got command BAM from MCS: ref #638616
2018-11-27 01:17:37 [INFO    ] Finished processing BAM request
2018-11-27 01:17:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:17:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638616   8 58449  4657033 A  ERROR'
2018-11-27 01:17:37 [DEBUG   ] Got command BAM from MCS: ref #638615
2018-11-27 01:17:37 [INFO    ] Finished processing BAM request
2018-11-27 01:17:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:17:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638615   8 58449  4657037 A  ERROR'
2018-11-27 01:17:37 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 01:17:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 01:17:37 [DEBUG   ] spawn_process: at 2018-11-27 01:17:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 01:17:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:17:53 [DEBUG   ] Got command RPT from MCS: ref #638617
2018-11-27 01:17:53 [DEBUG   ] summary = ERROR
2018-11-27 01:17:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638617  13 58449  4673598 A  ERRORERROR'
2018-11-27 01:17:53 [DEBUG   ] Got command RPT from MCS: ref #638618
2018-11-27 01:17:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:17:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638618  77 58449  4673789 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:17:53 [DEBUG   ] Got command RPT from MCS: ref #638619
2018-11-27 01:17:53 [DEBUG   ] summary = ERROR
2018-11-27 01:17:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638619  13 58449  4673993 A  ERRORERROR'
2018-11-27 01:17:54 [DEBUG   ] Got command RPT from MCS: ref #638620
2018-11-27 01:17:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:17:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638620  77 58449  4674197 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:17:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 01:17:54 [DEBUG   ] Got command RPT from MCS: ref #638621
2018-11-27 01:17:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:17:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638621  12 58449  4674401 A  ERRORB˜™š'
2018-11-27 01:17:54 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 01:17:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:17:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:17:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:17:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:17:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:17:54 [DEBUG   ] Finished updating board flags in 2.015 seconds
2018-11-27 01:17:57 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 01:18:17 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 01:18:37 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 01:18:39 [DEBUG   ] Got command BAM from MCS: ref #638622
2018-11-27 01:18:39 [INFO    ] Finished processing BAM request
2018-11-27 01:18:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:18:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638622   8 58449  4719025 A  ERROR'
2018-11-27 01:18:39 [DEBUG   ] Got command BAM from MCS: ref #638623
2018-11-27 01:18:39 [INFO    ] Finished processing BAM request
2018-11-27 01:18:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:18:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638623   8 58449  4719029 A  ERROR'
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 01:18:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 01:18:39 [DEBUG   ] spawn_process: at 2018-11-27 01:18:39 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 01:18:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:18:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:18:54 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:18:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 01:18:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:18:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:18:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:18:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:18:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:18:54 [DEBUG   ] Finished updating board flags in 1.990 seconds
2018-11-27 01:18:57 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 01:19:17 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 01:19:38 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 01:19:41 [DEBUG   ] Got command BAM from MCS: ref #638628
2018-11-27 01:19:41 [INFO    ] Finished processing BAM request
2018-11-27 01:19:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:19:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638628   8 58449  4781029 A  ERROR'
2018-11-27 01:19:41 [DEBUG   ] Got command BAM from MCS: ref #638629
2018-11-27 01:19:41 [INFO    ] Finished processing BAM request
2018-11-27 01:19:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:19:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638629   8 58449  4781033 A  ERROR'
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:19:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:19:41 [DEBUG   ] spawn_process: at 2018-11-27 01:19:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:19:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:19:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 01:19:54 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 01:19:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:19:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:19:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:19:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:19:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:19:54 [DEBUG   ] Finished updating board flags in 1.812 seconds
2018-11-27 01:19:58 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 01:20:18 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 01:20:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:20:43 [DEBUG   ] Got command BAM from MCS: ref #638630
2018-11-27 01:20:43 [INFO    ] Finished processing BAM request
2018-11-27 01:20:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:20:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638630   8 58449  4843025 A  ERROR'
2018-11-27 01:20:43 [DEBUG   ] Got command BAM from MCS: ref #638631
2018-11-27 01:20:43 [INFO    ] Finished processing BAM request
2018-11-27 01:20:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:20:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638631   8 58449  4843029 A  ERROR'
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:20:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:20:43 [DEBUG   ] spawn_process: at 2018-11-27 01:20:43 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:20:48 [DEBUG   ] Got command RPT from MCS: ref #638632
2018-11-27 01:20:48 [DEBUG   ] summary = ERROR
2018-11-27 01:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638632  13 58449  4848861 A  ERRORERROR'
2018-11-27 01:20:48 [DEBUG   ] Got command RPT from MCS: ref #638633
2018-11-27 01:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638633  77 58449  4848862 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:20:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:20:54 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 01:20:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:20:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:20:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:20:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:20:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:20:54 [DEBUG   ] Finished updating board flags in 1.794 seconds
2018-11-27 01:20:58 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2018-11-27 01:21:18 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 01:21:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:21:45 [DEBUG   ] Got command BAM from MCS: ref #638649
2018-11-27 01:21:45 [INFO    ] Finished processing BAM request
2018-11-27 01:21:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:21:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638649   8 58449  4905029 A  ERROR'
2018-11-27 01:21:45 [DEBUG   ] Got command BAM from MCS: ref #638650
2018-11-27 01:21:45 [INFO    ] Finished processing BAM request
2018-11-27 01:21:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:21:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638650   8 58449  4905033 A  ERROR'
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:21:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:21:45 [DEBUG   ] spawn_process: at 2018-11-27 01:21:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:21:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:21:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 01:21:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:21:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:21:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:21:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:21:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:21:54 [DEBUG   ] Finished updating board flags in 1.608 seconds
2018-11-27 01:21:58 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 01:22:18 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 01:22:38 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 01:22:47 [DEBUG   ] Got command BAM from MCS: ref #638657
2018-11-27 01:22:47 [INFO    ] Finished processing BAM request
2018-11-27 01:22:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:22:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638657   8 58449  4967033 A  ERROR'
2018-11-27 01:22:47 [DEBUG   ] Got command BAM from MCS: ref #638658
2018-11-27 01:22:47 [INFO    ] Finished processing BAM request
2018-11-27 01:22:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:22:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638658   8 58449  4967037 A  ERROR'
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 01:22:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 01:22:47 [DEBUG   ] spawn_process: at 2018-11-27 01:22:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 01:22:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:22:53 [DEBUG   ] Got command RPT from MCS: ref #638659
2018-11-27 01:22:53 [DEBUG   ] summary = ERROR
2018-11-27 01:22:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638659  13 58449  4973673 A  ERRORERROR'
2018-11-27 01:22:53 [DEBUG   ] Got command RPT from MCS: ref #638660
2018-11-27 01:22:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:22:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638660  77 58449  4973877 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:22:54 [DEBUG   ] Got command RPT from MCS: ref #638661
2018-11-27 01:22:54 [DEBUG   ] summary = ERROR
2018-11-27 01:22:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638661  13 58449  4974081 A  ERRORERROR'
2018-11-27 01:22:54 [DEBUG   ] Got command RPT from MCS: ref #638663
2018-11-27 01:22:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:22:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638663  77 58449  4974277 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:22:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 01:22:54 [DEBUG   ] Got command RPT from MCS: ref #638665
2018-11-27 01:22:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:22:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638665  12 58449  4974481 A  ERRORBŸff'
2018-11-27 01:22:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:22:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:22:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:22:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:22:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:22:54 [DEBUG   ] Finished updating board flags in 1.596 seconds
2018-11-27 01:22:59 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-27 01:23:19 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-27 01:23:39 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:23:49 [DEBUG   ] Got command BAM from MCS: ref #638669
2018-11-27 01:23:49 [INFO    ] Finished processing BAM request
2018-11-27 01:23:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:23:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638669   8 58449  5029034 A  ERROR'
2018-11-27 01:23:49 [DEBUG   ] Got command BAM from MCS: ref #638670
2018-11-27 01:23:49 [INFO    ] Finished processing BAM request
2018-11-27 01:23:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:23:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638670   8 58449  5029038 A  ERROR'
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:23:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:23:49 [DEBUG   ] spawn_process: at 2018-11-27 01:23:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:23:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2018-11-27 01:23:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:23:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:23:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:23:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:23:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:23:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:23:54 [DEBUG   ] Finished updating board flags in 1.415 seconds
2018-11-27 01:23:59 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:24:19 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:24:39 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:24:51 [DEBUG   ] Got command BAM from MCS: ref #638672
2018-11-27 01:24:51 [INFO    ] Finished processing BAM request
2018-11-27 01:24:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:24:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638672   8 58449  5091034 A  ERROR'
2018-11-27 01:24:51 [DEBUG   ] Got command BAM from MCS: ref #638671
2018-11-27 01:24:51 [INFO    ] Finished processing BAM request
2018-11-27 01:24:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:24:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638671   8 58449  5091038 A  ERROR'
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:24:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:24:51 [DEBUG   ] spawn_process: at 2018-11-27 01:24:51 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:24:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:24:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:24:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 01:24:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:24:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:24:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:24:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:24:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:24:54 [DEBUG   ] Finished updating board flags in 1.406 seconds
2018-11-27 01:24:59 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:25:19 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:25:39 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 01:25:48 [DEBUG   ] Got command RPT from MCS: ref #638677
2018-11-27 01:25:48 [DEBUG   ] summary = ERROR
2018-11-27 01:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638677  13 58449  5148865 A  ERRORERROR'
2018-11-27 01:25:48 [DEBUG   ] Got command RPT from MCS: ref #638678
2018-11-27 01:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638678  77 58449  5148866 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:25:53 [DEBUG   ] Got command BAM from MCS: ref #638694
2018-11-27 01:25:53 [INFO    ] Finished processing BAM request
2018-11-27 01:25:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:25:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638694   8 58449  5153033 A  ERROR'
2018-11-27 01:25:53 [DEBUG   ] Got command BAM from MCS: ref #638695
2018-11-27 01:25:53 [INFO    ] Finished processing BAM request
2018-11-27 01:25:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:25:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638695   8 58449  5153037 A  ERROR'
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:25:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:25:53 [DEBUG   ] spawn_process: at 2018-11-27 01:25:53 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:25:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:25:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:25:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:25:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:25:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:25:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:25:54 [DEBUG   ] Finished updating board flags in 1.455 seconds
2018-11-27 01:25:59 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:26:19 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 01:26:39 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:26:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:26:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:26:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:26:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:26:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:26:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:26:54 [DEBUG   ] Finished updating board flags in 1.475 seconds
2018-11-27 01:26:55 [DEBUG   ] Got command BAM from MCS: ref #638697
2018-11-27 01:26:55 [INFO    ] Finished processing BAM request
2018-11-27 01:26:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:26:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638697   8 58449  5215033 A  ERROR'
2018-11-27 01:26:55 [DEBUG   ] Got command BAM from MCS: ref #638696
2018-11-27 01:26:55 [INFO    ] Finished processing BAM request
2018-11-27 01:26:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:26:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638696   8 58449  5215037 A  ERROR'
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:26:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:26:55 [DEBUG   ] spawn_process: at 2018-11-27 01:26:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:26:59 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 01:27:19 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 01:27:39 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 01:27:53 [DEBUG   ] Got command RPT from MCS: ref #638698
2018-11-27 01:27:53 [DEBUG   ] summary = ERROR
2018-11-27 01:27:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638698  13 58449  5273765 A  ERRORERROR'
2018-11-27 01:27:53 [DEBUG   ] Got command RPT from MCS: ref #638699
2018-11-27 01:27:53 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:27:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638699  77 58449  5273969 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:27:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:27:54 [DEBUG   ] Got command RPT from MCS: ref #638700
2018-11-27 01:27:54 [DEBUG   ] summary = ERROR
2018-11-27 01:27:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638700  13 58449  5274166 A  ERRORERROR'
2018-11-27 01:27:54 [DEBUG   ] Got command RPT from MCS: ref #638701
2018-11-27 01:27:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:27:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638701  77 58449  5274369 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:27:54 [DEBUG   ] Got command RPT from MCS: ref #638702
2018-11-27 01:27:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:27:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638702  12 58449  5274573 A  ERRORB˜™š'
2018-11-27 01:27:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:27:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:27:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:27:54 [WARNING ] (13, 1, 'bot')
2018-11-27 01:27:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:27:54 [DEBUG   ] Finished updating board flags in 1.574 seconds
2018-11-27 01:27:57 [DEBUG   ] Got command BAM from MCS: ref #638703
2018-11-27 01:27:57 [INFO    ] Finished processing BAM request
2018-11-27 01:27:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:27:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638703   8 58449  5277033 A  ERROR'
2018-11-27 01:27:57 [DEBUG   ] Got command BAM from MCS: ref #638704
2018-11-27 01:27:57 [INFO    ] Finished processing BAM request
2018-11-27 01:27:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:27:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638704   8 58449  5277037 A  ERROR'
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:27:57 [DEBUG   ] spawn_process: at 2018-11-27 01:27:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:27:59 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:28:19 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:28:39 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 01:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 01:28:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2018-11-27 01:28:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:28:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:28:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:28:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:28:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:28:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:28:55 [DEBUG   ] Finished updating board flags in 2.218 seconds
2018-11-27 01:28:59 [DEBUG   ] Got command BAM from MCS: ref #638709
2018-11-27 01:28:59 [INFO    ] Finished processing BAM request
2018-11-27 01:28:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:28:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638709   8 58449  5339033 A  ERROR'
2018-11-27 01:28:59 [DEBUG   ] Got command BAM from MCS: ref #638710
2018-11-27 01:28:59 [INFO    ] Finished processing BAM request
2018-11-27 01:28:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:28:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638710   8 58449  5339037 A  ERROR'
2018-11-27 01:28:59 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:28:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:28:59 [DEBUG   ] spawn_process: at 2018-11-27 01:28:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:29:19 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 01:29:39 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 01:29:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:29:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:29:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:29:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:29:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:29:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:29:55 [DEBUG   ] Finished updating board flags in 2.078 seconds
2018-11-27 01:29:59 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 01:30:01 [DEBUG   ] Got command BAM from MCS: ref #638711
2018-11-27 01:30:01 [INFO    ] Finished processing BAM request
2018-11-27 01:30:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:30:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638711   8 58449  5401029 A  ERROR'
2018-11-27 01:30:01 [DEBUG   ] Got command BAM from MCS: ref #638712
2018-11-27 01:30:01 [INFO    ] Finished processing BAM request
2018-11-27 01:30:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:30:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638712   8 58449  5401033 A  ERROR'
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:30:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:30:01 [DEBUG   ] spawn_process: at 2018-11-27 01:30:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:30:19 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 01:30:39 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 01:30:48 [DEBUG   ] Got command RPT from MCS: ref #638713
2018-11-27 01:30:48 [DEBUG   ] summary = ERROR
2018-11-27 01:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638713  13 58449  5448865 A  ERRORERROR'
2018-11-27 01:30:48 [DEBUG   ] Got command RPT from MCS: ref #638714
2018-11-27 01:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638714  77 58449  5448866 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:30:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:30:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:30:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:30:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:30:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:30:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:30:55 [DEBUG   ] Finished updating board flags in 2.015 seconds
2018-11-27 01:31:00 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 01:31:03 [DEBUG   ] Got command BAM from MCS: ref #638730
2018-11-27 01:31:03 [INFO    ] Finished processing BAM request
2018-11-27 01:31:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:31:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638730   8 58449  5463033 A  ERROR'
2018-11-27 01:31:03 [DEBUG   ] Got command BAM from MCS: ref #638731
2018-11-27 01:31:03 [INFO    ] Finished processing BAM request
2018-11-27 01:31:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:31:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638731   8 58449  5463037 A  ERROR'
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:31:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:31:03 [DEBUG   ] spawn_process: at 2018-11-27 01:31:03 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:31:19 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-27 01:31:40 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2018-11-27 01:31:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 01:31:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:31:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:31:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:31:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:31:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:31:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:31:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:31:55 [DEBUG   ] Finished updating board flags in 1.997 seconds
2018-11-27 01:32:00 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-27 01:32:05 [DEBUG   ] Got command BAM from MCS: ref #638736
2018-11-27 01:32:05 [INFO    ] Finished processing BAM request
2018-11-27 01:32:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:32:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638736   8 58449  5525025 A  ERROR'
2018-11-27 01:32:05 [DEBUG   ] Got command BAM from MCS: ref #638737
2018-11-27 01:32:05 [INFO    ] Finished processing BAM request
2018-11-27 01:32:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:32:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638737   8 58449  5525029 A  ERROR'
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:32:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:32:05 [DEBUG   ] spawn_process: at 2018-11-27 01:32:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:32:20 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:32:40 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2018-11-27 01:32:53 [DEBUG   ] Got command RPT from MCS: ref #638740
2018-11-27 01:32:53 [DEBUG   ] summary = ERROR
2018-11-27 01:32:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638740  13 58449  5573849 A  ERRORERROR'
2018-11-27 01:32:54 [DEBUG   ] Got command RPT from MCS: ref #638741
2018-11-27 01:32:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:32:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638741  77 58449  5574053 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:32:54 [DEBUG   ] Got command RPT from MCS: ref #638742
2018-11-27 01:32:54 [DEBUG   ] summary = ERROR
2018-11-27 01:32:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638742  13 58449  5574249 A  ERRORERROR'
2018-11-27 01:32:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 01:32:54 [DEBUG   ] Got command RPT from MCS: ref #638744
2018-11-27 01:32:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:32:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638744  77 58449  5574453 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:32:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:32:54 [DEBUG   ] Got command RPT from MCS: ref #638745
2018-11-27 01:32:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:32:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638745  12 58449  5574662 A  ERRORB–™š'
2018-11-27 01:32:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:32:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:32:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:32:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:32:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:32:55 [DEBUG   ] Finished updating board flags in 1.810 seconds
2018-11-27 01:33:00 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:33:07 [DEBUG   ] Got command BAM from MCS: ref #638750
2018-11-27 01:33:07 [INFO    ] Finished processing BAM request
2018-11-27 01:33:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:33:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638750   8 58449  5587033 A  ERROR'
2018-11-27 01:33:07 [DEBUG   ] Got command BAM from MCS: ref #638751
2018-11-27 01:33:07 [INFO    ] Finished processing BAM request
2018-11-27 01:33:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:33:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638751   8 58449  5587037 A  ERROR'
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:33:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:33:07 [DEBUG   ] spawn_process: at 2018-11-27 01:33:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:33:20 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:33:40 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 01:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 01:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 01:33:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:33:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:33:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 01:33:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:33:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:33:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:33:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:33:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:33:55 [DEBUG   ] Finished updating board flags in 1.793 seconds
2018-11-27 01:34:00 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 01:34:09 [DEBUG   ] Got command BAM from MCS: ref #638756
2018-11-27 01:34:09 [INFO    ] Finished processing BAM request
2018-11-27 01:34:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:34:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638756   8 58449  5649033 A  ERROR'
2018-11-27 01:34:09 [DEBUG   ] Got command BAM from MCS: ref #638757
2018-11-27 01:34:09 [INFO    ] Finished processing BAM request
2018-11-27 01:34:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:34:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638757   8 58449  5649041 A  ERROR'
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:34:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:34:09 [DEBUG   ] spawn_process: at 2018-11-27 01:34:09 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:34:20 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:34:40 [DEBUG   ] Finished updating ant stats in 0.794 seconds
2018-11-27 01:34:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 01:34:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:34:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 01:34:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:34:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:34:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:34:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:34:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:34:55 [DEBUG   ] Finished updating board flags in 1.602 seconds
2018-11-27 01:35:00 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 01:35:11 [DEBUG   ] Got command BAM from MCS: ref #638759
2018-11-27 01:35:11 [INFO    ] Finished processing BAM request
2018-11-27 01:35:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:35:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638759   8 58449  5711029 A  ERROR'
2018-11-27 01:35:11 [DEBUG   ] Got command BAM from MCS: ref #638758
2018-11-27 01:35:11 [INFO    ] Finished processing BAM request
2018-11-27 01:35:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:35:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638758   8 58449  5711033 A  ERROR'
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:35:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:35:11 [DEBUG   ] spawn_process: at 2018-11-27 01:35:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:35:21 [DEBUG   ] Finished updating ant stats in 1.598 seconds
2018-11-27 01:35:40 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 01:35:48 [DEBUG   ] Got command RPT from MCS: ref #638760
2018-11-27 01:35:48 [DEBUG   ] summary = ERROR
2018-11-27 01:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638760  13 58449  5748869 A  ERRORERROR'
2018-11-27 01:35:48 [DEBUG   ] Got command RPT from MCS: ref #638761
2018-11-27 01:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638761  77 58449  5748870 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:35:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:35:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:35:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 01:35:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:35:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:35:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:35:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:35:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:35:55 [DEBUG   ] Finished updating board flags in 1.607 seconds
2018-11-27 01:36:01 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2018-11-27 01:36:14 [DEBUG   ] Got command BAM from MCS: ref #638778
2018-11-27 01:36:14 [INFO    ] Finished processing BAM request
2018-11-27 01:36:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:36:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638778   8 58449  5774025 A  ERROR'
2018-11-27 01:36:14 [DEBUG   ] Got command BAM from MCS: ref #638777
2018-11-27 01:36:14 [INFO    ] Finished processing BAM request
2018-11-27 01:36:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:36:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638777   8 58449  5774029 A  ERROR'
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 01:36:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 01:36:14 [DEBUG   ] spawn_process: at 2018-11-27 01:36:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 01:36:20 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-27 01:36:41 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2018-11-27 01:36:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:36:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:36:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:36:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:36:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:36:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:36:55 [DEBUG   ] Finished updating board flags in 1.596 seconds
2018-11-27 01:37:01 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2018-11-27 01:37:17 [DEBUG   ] Got command BAM from MCS: ref #638783
2018-11-27 01:37:17 [INFO    ] Finished processing BAM request
2018-11-27 01:37:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:37:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638783   8 58449  5837025 A  ERROR'
2018-11-27 01:37:17 [DEBUG   ] Got command BAM from MCS: ref #638784
2018-11-27 01:37:17 [INFO    ] Finished processing BAM request
2018-11-27 01:37:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:37:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638784   8 58449  5837029 A  ERROR'
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:37:17 [DEBUG   ] spawn_process: at 2018-11-27 01:37:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:37:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 01:37:41 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 01:37:53 [DEBUG   ] Got command RPT from MCS: ref #638785
2018-11-27 01:37:53 [DEBUG   ] summary = ERROR
2018-11-27 01:37:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638785  13 58449  5873941 A  ERRORERROR'
2018-11-27 01:37:54 [DEBUG   ] Got command RPT from MCS: ref #638786
2018-11-27 01:37:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:37:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638786  77 58449  5874145 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:37:54 [DEBUG   ] Got command RPT from MCS: ref #638787
2018-11-27 01:37:54 [DEBUG   ] summary = ERROR
2018-11-27 01:37:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638787  13 58449  5874349 A  ERRORERROR'
2018-11-27 01:37:54 [DEBUG   ] Got command RPT from MCS: ref #638788
2018-11-27 01:37:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:37:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638788  77 58449  5874541 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:37:54 [DEBUG   ] Got command RPT from MCS: ref #638789
2018-11-27 01:37:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:37:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638789  12 58449  5874745 A  ERRORB–™š'
2018-11-27 01:37:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:37:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:37:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:37:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:37:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:37:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:37:55 [DEBUG   ] Finished updating board flags in 1.398 seconds
2018-11-27 01:38:01 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:38:20 [DEBUG   ] Got command BAM from MCS: ref #638791
2018-11-27 01:38:20 [INFO    ] Finished processing BAM request
2018-11-27 01:38:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:38:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638791   8 58449  5900033 A  ERROR'
2018-11-27 01:38:20 [DEBUG   ] Got command BAM from MCS: ref #638790
2018-11-27 01:38:20 [INFO    ] Finished processing BAM request
2018-11-27 01:38:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:38:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638790   8 58449  5900037 A  ERROR'
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 01:38:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 01:38:20 [DEBUG   ] spawn_process: at 2018-11-27 01:38:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 01:38:21 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 01:38:41 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 01:38:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:38:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:38:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:38:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:38:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:38:55 [WARNING ] (13, 1, 'bot')
2018-11-27 01:38:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:38:55 [DEBUG   ] Finished updating board flags in 1.344 seconds
2018-11-27 01:39:01 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 01:39:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:39:23 [DEBUG   ] Got command BAM from MCS: ref #638793
2018-11-27 01:39:23 [INFO    ] Finished processing BAM request
2018-11-27 01:39:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:39:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638793   8 58449  5963025 A  ERROR'
2018-11-27 01:39:23 [DEBUG   ] Got command BAM from MCS: ref #638792
2018-11-27 01:39:23 [INFO    ] Finished processing BAM request
2018-11-27 01:39:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:39:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638792   8 58449  5963029 A  ERROR'
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 01:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 01:39:23 [DEBUG   ] spawn_process: at 2018-11-27 01:39:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 01:39:41 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 01:39:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:39:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:39:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:39:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:39:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:39:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:39:56 [DEBUG   ] Finished updating board flags in 2.081 seconds
2018-11-27 01:40:01 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 01:40:21 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 01:40:26 [DEBUG   ] Got command BAM from MCS: ref #638799
2018-11-27 01:40:26 [INFO    ] Finished processing BAM request
2018-11-27 01:40:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:40:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638799   8 58449  6026025 A  ERROR'
2018-11-27 01:40:26 [DEBUG   ] Got command BAM from MCS: ref #638798
2018-11-27 01:40:26 [INFO    ] Finished processing BAM request
2018-11-27 01:40:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:40:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638798   8 58449  6026029 A  ERROR'
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:40:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:40:26 [DEBUG   ] spawn_process: at 2018-11-27 01:40:26 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:40:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 01:40:48 [DEBUG   ] Got command RPT from MCS: ref #638800
2018-11-27 01:40:48 [DEBUG   ] summary = ERROR
2018-11-27 01:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638800  13 58449  6048865 A  ERRORERROR'
2018-11-27 01:40:48 [DEBUG   ] Got command RPT from MCS: ref #638801
2018-11-27 01:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638801  77 58449  6048866 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:40:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:40:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:40:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:40:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:40:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:40:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:40:56 [DEBUG   ] Finished updating board flags in 1.977 seconds
2018-11-27 01:41:01 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 01:41:21 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 01:41:29 [DEBUG   ] Got command BAM from MCS: ref #638817
2018-11-27 01:41:29 [INFO    ] Finished processing BAM request
2018-11-27 01:41:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:41:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638817   8 58449  6089029 A  ERROR'
2018-11-27 01:41:29 [DEBUG   ] Got command BAM from MCS: ref #638818
2018-11-27 01:41:29 [INFO    ] Finished processing BAM request
2018-11-27 01:41:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:41:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638818   8 58449  6089033 A  ERROR'
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:41:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:41:29 [DEBUG   ] spawn_process: at 2018-11-27 01:41:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:41:42 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-27 01:41:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 01:41:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:41:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:41:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:41:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:41:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:41:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:41:56 [DEBUG   ] Finished updating board flags in 1.810 seconds
2018-11-27 01:42:02 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2018-11-27 01:42:22 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 01:42:32 [DEBUG   ] Got command BAM from MCS: ref #638819
2018-11-27 01:42:32 [INFO    ] Finished processing BAM request
2018-11-27 01:42:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:42:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638819   8 58449  6152025 A  ERROR'
2018-11-27 01:42:32 [DEBUG   ] Got command BAM from MCS: ref #638820
2018-11-27 01:42:32 [INFO    ] Finished processing BAM request
2018-11-27 01:42:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:42:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638820   8 58449  6152029 A  ERROR'
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:42:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:42:32 [DEBUG   ] spawn_process: at 2018-11-27 01:42:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:42:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:42:54 [DEBUG   ] Got command RPT from MCS: ref #638823
2018-11-27 01:42:54 [DEBUG   ] summary = ERROR
2018-11-27 01:42:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638823  13 58449  6174025 A  ERRORERROR'
2018-11-27 01:42:54 [DEBUG   ] Got command RPT from MCS: ref #638824
2018-11-27 01:42:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:42:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638824  77 58449  6174229 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:42:54 [DEBUG   ] Got command RPT from MCS: ref #638825
2018-11-27 01:42:54 [DEBUG   ] summary = ERROR
2018-11-27 01:42:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638825  13 58449  6174433 A  ERRORERROR'
2018-11-27 01:42:54 [DEBUG   ] Got command RPT from MCS: ref #638827
2018-11-27 01:42:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:42:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638827  77 58449  6174625 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:42:54 [DEBUG   ] Got command RPT from MCS: ref #638829
2018-11-27 01:42:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:42:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638829  12 58449  6174829 A  ERRORBŸff'
2018-11-27 01:42:55 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:42:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 01:42:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:42:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 01:42:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:42:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:42:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:42:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:42:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:42:56 [DEBUG   ] Finished updating board flags in 1.796 seconds
2018-11-27 01:43:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:43:22 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 01:43:32 [DEBUG   ] Got command STP from MCS: ref #638838
2018-11-27 01:43:32 [INFO    ] Finished processing BAM request
2018-11-27 01:43:32 [INFO    ] Shutdown the adder chain on beam #4
2018-11-27 01:43:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   638838   8 58449  6212029 A  ERROR'
2018-11-27 01:43:32 [DEBUG   ] Got command STP from MCS: ref #638837
2018-11-27 01:43:32 [INFO    ] Finished processing BAM request
2018-11-27 01:43:32 [INFO    ] Shutdown the adder chain on beam #3
2018-11-27 01:43:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   638837   8 58449  6212033 A  ERROR'
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 01:43:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 01:43:32 [DEBUG   ] spawn_process: at 2018-11-27 01:43:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 01:43:42 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 01:43:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:43:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:43:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:43:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:43:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:43:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:43:56 [DEBUG   ] Finished updating board flags in 1.611 seconds
2018-11-27 01:44:02 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:44:22 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:44:42 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2018-11-27 01:44:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:44:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 01:44:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:44:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:44:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:44:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:44:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:44:56 [DEBUG   ] Finished updating board flags in 1.594 seconds
2018-11-27 01:45:02 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 01:45:22 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 01:45:42 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 01:45:48 [DEBUG   ] Got command RPT from MCS: ref #638839
2018-11-27 01:45:48 [DEBUG   ] summary = ERROR
2018-11-27 01:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638839  13 58449  6348866 A  ERRORERROR'
2018-11-27 01:45:48 [DEBUG   ] Got command RPT from MCS: ref #638840
2018-11-27 01:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638840  77 58449  6348866 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:45:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:45:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:45:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:45:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:45:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:45:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:45:56 [DEBUG   ] Finished updating board flags in 1.410 seconds
2018-11-27 01:46:02 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 01:46:22 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 01:46:43 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 01:46:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:46:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:46:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:46:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:46:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:46:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:46:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:46:56 [DEBUG   ] Finished updating board flags in 1.386 seconds
2018-11-27 01:47:03 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-27 01:47:23 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:47:43 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:47:54 [DEBUG   ] Got command RPT from MCS: ref #638860
2018-11-27 01:47:54 [DEBUG   ] summary = ERROR
2018-11-27 01:47:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638860  13 58449  6474110 A  ERRORERROR'
2018-11-27 01:47:54 [DEBUG   ] Got command RPT from MCS: ref #638861
2018-11-27 01:47:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:47:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638861  77 58449  6474313 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:47:54 [DEBUG   ] Got command RPT from MCS: ref #638862
2018-11-27 01:47:54 [DEBUG   ] summary = ERROR
2018-11-27 01:47:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638862  13 58449  6474517 A  ERRORERROR'
2018-11-27 01:47:54 [DEBUG   ] Got command RPT from MCS: ref #638863
2018-11-27 01:47:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:47:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638863  77 58449  6474721 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:47:54 [DEBUG   ] Got command RPT from MCS: ref #638864
2018-11-27 01:47:54 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:47:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638864  12 58449  6474914 A  ERRORB™™š'
2018-11-27 01:47:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:47:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:47:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:47:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:47:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:47:56 [WARNING ] (13, 1, 'bot')
2018-11-27 01:47:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:47:56 [DEBUG   ] Finished updating board flags in 1.356 seconds
2018-11-27 01:48:03 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:48:23 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:48:43 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 01:48:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:48:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:48:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:48:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:48:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:48:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:48:57 [DEBUG   ] Finished updating board flags in 2.112 seconds
2018-11-27 01:49:03 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:49:23 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 01:49:43 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 01:49:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:49:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:49:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:49:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:49:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:49:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:49:57 [DEBUG   ] Finished updating board flags in 1.953 seconds
2018-11-27 01:50:03 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 01:50:23 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:50:43 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 01:50:48 [DEBUG   ] Got command RPT from MCS: ref #638869
2018-11-27 01:50:48 [DEBUG   ] summary = ERROR
2018-11-27 01:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638869  13 58449  6648869 A  ERRORERROR'
2018-11-27 01:50:48 [DEBUG   ] Got command RPT from MCS: ref #638870
2018-11-27 01:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638870  77 58449  6648870 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:50:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:50:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:50:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:50:57 [DEBUG   ] Finished updating board flags in 1.808 seconds
2018-11-27 01:51:03 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 01:51:23 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 01:51:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 01:51:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:51:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:51:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:51:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:51:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:51:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:51:57 [DEBUG   ] Finished updating board flags in 1.804 seconds
2018-11-27 01:52:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 01:52:23 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 01:52:43 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 01:52:54 [DEBUG   ] Got command RPT from MCS: ref #638892
2018-11-27 01:52:54 [DEBUG   ] summary = ERROR
2018-11-27 01:52:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638892  13 58449  6774201 A  ERRORERROR'
2018-11-27 01:52:54 [DEBUG   ] Got command RPT from MCS: ref #638893
2018-11-27 01:52:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:52:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638893  77 58449  6774405 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:52:54 [DEBUG   ] Got command RPT from MCS: ref #638894
2018-11-27 01:52:54 [DEBUG   ] summary = ERROR
2018-11-27 01:52:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638894  13 58449  6774597 A  ERRORERROR'
2018-11-27 01:52:54 [DEBUG   ] Got command RPT from MCS: ref #638896
2018-11-27 01:52:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:52:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638896  77 58449  6774801 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:52:55 [DEBUG   ] Got command RPT from MCS: ref #638898
2018-11-27 01:52:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:52:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638898  12 58449  6775005 A  ERRORB—™š'
2018-11-27 01:52:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:52:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:52:57 [DEBUG   ] Finished updating board flags in 1.838 seconds
2018-11-27 01:53:03 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 01:53:24 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 01:53:44 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 01:53:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:53:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:53:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:53:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:53:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:53:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:53:57 [DEBUG   ] Finished updating board flags in 1.754 seconds
2018-11-27 01:54:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:54:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:54:31 [DEBUG   ] Got command DRX from MCS: ref #638906
2018-11-27 01:54:31 [INFO    ] sending drx command to lwa16
2018-11-27 01:54:31 [DEBUG   ] run_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 35100000.0 7 9 0']
2018-11-27 01:54:31 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 01:54:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   638906   8 58449  6871009 A  ERROR'
2018-11-27 01:54:31 [DEBUG   ] Got command DRX from MCS: ref #638907
2018-11-27 01:54:31 [INFO    ] sending drx command to lwa16
2018-11-27 01:54:31 [DEBUG   ] run_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 64500000.0 7 9 0']
2018-11-27 01:54:31 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 01:54:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   638907   8 58449  6871013 A  ERROR'
2018-11-27 01:54:31 [DEBUG   ] Got command DRX from MCS: ref #638909
2018-11-27 01:54:31 [INFO    ] sending drx command to lwa16
2018-11-27 01:54:31 [DEBUG   ] run_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 79200000.0 7 9 0']
2018-11-27 01:54:31 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 01:54:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   638909   8 58449  6871017 A  ERROR'
2018-11-27 01:54:31 [DEBUG   ] Got command DRX from MCS: ref #638908
2018-11-27 01:54:31 [INFO    ] sending drx command to lwa16
2018-11-27 01:54:31 [DEBUG   ] run_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 49800000.0 7 9 0']
2018-11-27 01:54:31 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 01:54:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   638908   8 58449  6871021 A  ERROR'
2018-11-27 01:54:31 [DEBUG   ] Got command BAM from MCS: ref #638911
2018-11-27 01:54:31 [INFO    ] Finished processing BAM request
2018-11-27 01:54:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:54:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638911   8 58449  6871025 A  ERROR'
2018-11-27 01:54:31 [DEBUG   ] Got command BAM from MCS: ref #638910
2018-11-27 01:54:31 [INFO    ] Finished processing BAM request
2018-11-27 01:54:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:54:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638910   8 58449  6871029 A  ERROR'
2018-11-27 01:54:31 [INFO    ] Finished starting DRX
2018-11-27 01:54:31 [INFO    ] Finished starting DRX
2018-11-27 01:54:31 [INFO    ] Finished starting DRX
2018-11-27 01:54:31 [INFO    ] Finished starting DRX
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:54:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:54:31 [DEBUG   ] spawn_process: at 2018-11-27 01:54:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:54:44 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:54:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 01:54:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:54:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-27 01:54:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:54:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:54:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:54:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:54:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:54:57 [DEBUG   ] Finished updating board flags in 1.617 seconds
2018-11-27 01:55:04 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 01:55:21 [DEBUG   ] Got command BAM from MCS: ref #638918
2018-11-27 01:55:21 [INFO    ] Finished processing BAM request
2018-11-27 01:55:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:55:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638918   8 58449  6921025 A  ERROR'
2018-11-27 01:55:21 [DEBUG   ] Got command BAM from MCS: ref #638919
2018-11-27 01:55:21 [INFO    ] Finished processing BAM request
2018-11-27 01:55:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:55:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638919   8 58449  6921029 A  ERROR'
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 01:55:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 01:55:21 [DEBUG   ] spawn_process: at 2018-11-27 01:55:21 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 01:55:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:55:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:55:48 [DEBUG   ] Got command RPT from MCS: ref #638920
2018-11-27 01:55:48 [DEBUG   ] summary = ERROR
2018-11-27 01:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638920  13 58449  6948869 A  ERRORERROR'
2018-11-27 01:55:48 [DEBUG   ] Got command RPT from MCS: ref #638921
2018-11-27 01:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638921  77 58449  6948870 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:55:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:55:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:55:57 [DEBUG   ] Finished updating board flags in 1.620 seconds
2018-11-27 01:56:04 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:56:11 [DEBUG   ] Got command BAM from MCS: ref #638938
2018-11-27 01:56:11 [INFO    ] Finished processing BAM request
2018-11-27 01:56:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:56:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638938   8 58449  6971033 A  ERROR'
2018-11-27 01:56:11 [DEBUG   ] Got command BAM from MCS: ref #638937
2018-11-27 01:56:11 [INFO    ] Finished processing BAM request
2018-11-27 01:56:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:56:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638937   8 58449  6971037 A  ERROR'
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 01:56:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 01:56:11 [DEBUG   ] spawn_process: at 2018-11-27 01:56:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 01:56:24 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 01:56:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 01:56:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:56:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:56:57 [DEBUG   ] Finished updating board flags in 1.640 seconds
2018-11-27 01:57:01 [DEBUG   ] Got command BAM from MCS: ref #638940
2018-11-27 01:57:01 [INFO    ] Finished processing BAM request
2018-11-27 01:57:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:57:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638940   8 58449  7021033 A  ERROR'
2018-11-27 01:57:01 [DEBUG   ] Got command BAM from MCS: ref #638939
2018-11-27 01:57:01 [INFO    ] Finished processing BAM request
2018-11-27 01:57:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:57:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638939   8 58449  7021037 A  ERROR'
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 01:57:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 01:57:01 [DEBUG   ] spawn_process: at 2018-11-27 01:57:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 01:57:04 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 01:57:24 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 01:57:44 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 01:57:51 [DEBUG   ] Got command BAM from MCS: ref #638941
2018-11-27 01:57:51 [INFO    ] Finished processing BAM request
2018-11-27 01:57:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:57:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638941   8 58449  7071025 A  ERROR'
2018-11-27 01:57:51 [DEBUG   ] Got command BAM from MCS: ref #638942
2018-11-27 01:57:51 [INFO    ] Finished processing BAM request
2018-11-27 01:57:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:57:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638942   8 58449  7071029 A  ERROR'
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 01:57:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 01:57:51 [DEBUG   ] spawn_process: at 2018-11-27 01:57:51 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 01:57:54 [DEBUG   ] Got command RPT from MCS: ref #638943
2018-11-27 01:57:54 [DEBUG   ] summary = ERROR
2018-11-27 01:57:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638943  13 58449  7074289 A  ERRORERROR'
2018-11-27 01:57:54 [DEBUG   ] Got command RPT from MCS: ref #638944
2018-11-27 01:57:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:57:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638944  77 58449  7074493 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:57:54 [DEBUG   ] Got command RPT from MCS: ref #638945
2018-11-27 01:57:54 [DEBUG   ] summary = ERROR
2018-11-27 01:57:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638945  13 58449  7074685 A  ERRORERROR'
2018-11-27 01:57:54 [DEBUG   ] Got command RPT from MCS: ref #638946
2018-11-27 01:57:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 01:57:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638946  77 58449  7074889 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 01:57:55 [DEBUG   ] Got command RPT from MCS: ref #638947
2018-11-27 01:57:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 01:57:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638947  12 58449  7075093 A  ERRORBžff'
2018-11-27 01:57:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 01:57:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:57:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:57:57 [DEBUG   ] Finished updating board flags in 1.534 seconds
2018-11-27 01:58:04 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:58:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:58:41 [DEBUG   ] Got command BAM from MCS: ref #638952
2018-11-27 01:58:41 [INFO    ] Finished processing BAM request
2018-11-27 01:58:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:58:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638952   8 58449  7121025 A  ERROR'
2018-11-27 01:58:41 [DEBUG   ] Got command BAM from MCS: ref #638953
2018-11-27 01:58:41 [INFO    ] Finished processing BAM request
2018-11-27 01:58:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:58:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638953   8 58449  7121029 A  ERROR'
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:58:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:58:41 [DEBUG   ] spawn_process: at 2018-11-27 01:58:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:58:44 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 01:58:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 01:58:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:58:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 01:58:57 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 01:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:58:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:58:57 [DEBUG   ] Finished updating board flags in 1.395 seconds
2018-11-27 01:59:04 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 01:59:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:59:31 [DEBUG   ] Got command BAM from MCS: ref #638955
2018-11-27 01:59:31 [INFO    ] Finished processing BAM request
2018-11-27 01:59:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:59:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638955   8 58449  7171025 A  ERROR'
2018-11-27 01:59:31 [DEBUG   ] Got command BAM from MCS: ref #638954
2018-11-27 01:59:31 [INFO    ] Finished processing BAM request
2018-11-27 01:59:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 01:59:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638954   8 58449  7171029 A  ERROR'
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 01:59:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 01:59:31 [DEBUG   ] spawn_process: at 2018-11-27 01:59:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 01:59:44 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 01:59:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 01:59:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 01:59:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:59:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:59:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:59:57 [WARNING ] (13, 1, 'bot')
2018-11-27 01:59:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 01:59:57 [DEBUG   ] Finished updating board flags in 1.489 seconds
2018-11-27 02:00:04 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 02:00:20 [DEBUG   ] Got command BAM from MCS: ref #638956
2018-11-27 02:00:20 [INFO    ] Finished processing BAM request
2018-11-27 02:00:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:00:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638956   8 58449  7220033 A  ERROR'
2018-11-27 02:00:20 [DEBUG   ] Got command BAM from MCS: ref #638957
2018-11-27 02:00:20 [INFO    ] Finished processing BAM request
2018-11-27 02:00:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:00:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638957   8 58449  7220037 A  ERROR'
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:00:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:00:20 [DEBUG   ] spawn_process: at 2018-11-27 02:00:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:00:24 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 02:00:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:00:48 [DEBUG   ] Got command RPT from MCS: ref #638958
2018-11-27 02:00:48 [DEBUG   ] summary = ERROR
2018-11-27 02:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638958  13 58449  7248869 A  ERRORERROR'
2018-11-27 02:00:48 [DEBUG   ] Got command RPT from MCS: ref #638959
2018-11-27 02:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638959  77 58449  7248870 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:00:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:00:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 02:00:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:00:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:00:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:00:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:00:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:00:58 [DEBUG   ] Finished updating board flags in 2.193 seconds
2018-11-27 02:01:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:01:09 [DEBUG   ] Got command BAM from MCS: ref #638979
2018-11-27 02:01:09 [INFO    ] Finished processing BAM request
2018-11-27 02:01:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:01:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638979   8 58449  7269033 A  ERROR'
2018-11-27 02:01:09 [DEBUG   ] Got command BAM from MCS: ref #638980
2018-11-27 02:01:09 [INFO    ] Finished processing BAM request
2018-11-27 02:01:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:01:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638980   8 58449  7269037 A  ERROR'
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:01:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:01:09 [DEBUG   ] spawn_process: at 2018-11-27 02:01:09 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:01:24 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 02:01:44 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 02:01:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 02:01:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:01:58 [DEBUG   ] Got command BAM from MCS: ref #638981
2018-11-27 02:01:58 [INFO    ] Finished processing BAM request
2018-11-27 02:01:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638981   8 58449  7318041 A  ERROR'
2018-11-27 02:01:58 [DEBUG   ] Got command BAM from MCS: ref #638982
2018-11-27 02:01:58 [INFO    ] Finished processing BAM request
2018-11-27 02:01:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:01:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638982   8 58449  7318049 A  ERROR'
2018-11-27 02:01:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:01:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:01:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:01:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:01:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:01:58 [DEBUG   ] Finished updating board flags in 1.939 seconds
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:01:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:01:58 [DEBUG   ] spawn_process: at 2018-11-27 02:01:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:02:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:02:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:02:44 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:02:47 [DEBUG   ] Got command BAM from MCS: ref #638985
2018-11-27 02:02:47 [INFO    ] Finished processing BAM request
2018-11-27 02:02:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:02:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638985   8 58449  7367029 A  ERROR'
2018-11-27 02:02:47 [DEBUG   ] Got command BAM from MCS: ref #638986
2018-11-27 02:02:47 [INFO    ] Finished processing BAM request
2018-11-27 02:02:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:02:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638986   8 58449  7367033 A  ERROR'
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:02:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:02:47 [DEBUG   ] spawn_process: at 2018-11-27 02:02:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:02:54 [DEBUG   ] Got command RPT from MCS: ref #638987
2018-11-27 02:02:54 [DEBUG   ] summary = ERROR
2018-11-27 02:02:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638987  13 58449  7374377 A  ERRORERROR'
2018-11-27 02:02:54 [DEBUG   ] Got command RPT from MCS: ref #638988
2018-11-27 02:02:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:02:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638988  77 58449  7374569 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:02:54 [DEBUG   ] Got command RPT from MCS: ref #638989
2018-11-27 02:02:54 [DEBUG   ] summary = ERROR
2018-11-27 02:02:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638989  13 58449  7374773 A  ERRORERROR'
2018-11-27 02:02:54 [DEBUG   ] Got command RPT from MCS: ref #638991
2018-11-27 02:02:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:02:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638991  77 58449  7374977 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:02:55 [DEBUG   ] Got command RPT from MCS: ref #638992
2018-11-27 02:02:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:02:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   638992  12 58449  7375181 A  ERRORBžff'
2018-11-27 02:02:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:02:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:02:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:02:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:02:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:02:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:02:58 [DEBUG   ] Finished updating board flags in 1.789 seconds
2018-11-27 02:03:04 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2018-11-27 02:03:24 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 02:03:36 [DEBUG   ] Got command BAM from MCS: ref #638998
2018-11-27 02:03:36 [INFO    ] Finished processing BAM request
2018-11-27 02:03:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:03:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638998   8 58449  7416029 A  ERROR'
2018-11-27 02:03:36 [DEBUG   ] Got command BAM from MCS: ref #638997
2018-11-27 02:03:36 [INFO    ] Finished processing BAM request
2018-11-27 02:03:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:03:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   638997   8 58449  7416033 A  ERROR'
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:03:36 [DEBUG   ] spawn_process: at 2018-11-27 02:03:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:03:45 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 02:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 02:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 02:03:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:03:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 02:03:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 02:03:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 02:03:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:03:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:03:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:03:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:03:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:03:58 [DEBUG   ] Finished updating board flags in 1.609 seconds
2018-11-27 02:04:05 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-27 02:04:25 [DEBUG   ] Got command BAM from MCS: ref #639004
2018-11-27 02:04:25 [INFO    ] Finished processing BAM request
2018-11-27 02:04:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:04:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639004   8 58449  7465029 A  ERROR'
2018-11-27 02:04:25 [DEBUG   ] Got command BAM from MCS: ref #639003
2018-11-27 02:04:25 [INFO    ] Finished processing BAM request
2018-11-27 02:04:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:04:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639003   8 58449  7465033 A  ERROR'
2018-11-27 02:04:25 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:04:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:04:25 [DEBUG   ] spawn_process: at 2018-11-27 02:04:25 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:04:45 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:04:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:04:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:04:58 [DEBUG   ] Finished updating board flags in 1.595 seconds
2018-11-27 02:05:05 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:05:14 [DEBUG   ] Got command BAM from MCS: ref #639006
2018-11-27 02:05:14 [INFO    ] Finished processing BAM request
2018-11-27 02:05:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:05:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639006   8 58449  7514034 A  ERROR'
2018-11-27 02:05:14 [DEBUG   ] Got command BAM from MCS: ref #639005
2018-11-27 02:05:14 [INFO    ] Finished processing BAM request
2018-11-27 02:05:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:05:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639005   8 58449  7514038 A  ERROR'
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:05:14 [DEBUG   ] spawn_process: at 2018-11-27 02:05:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:05:25 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2018-11-27 02:05:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 02:05:48 [DEBUG   ] Got command RPT from MCS: ref #639007
2018-11-27 02:05:48 [DEBUG   ] summary = ERROR
2018-11-27 02:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639007  13 58449  7548873 A  ERRORERROR'
2018-11-27 02:05:48 [DEBUG   ] Got command RPT from MCS: ref #639008
2018-11-27 02:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639008  77 58449  7548874 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:05:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:05:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:05:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:05:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:05:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:05:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:05:58 [DEBUG   ] Finished updating board flags in 1.410 seconds
2018-11-27 02:06:03 [DEBUG   ] Got command BAM from MCS: ref #639024
2018-11-27 02:06:03 [INFO    ] Finished processing BAM request
2018-11-27 02:06:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:06:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639024   8 58449  7563026 A  ERROR'
2018-11-27 02:06:03 [DEBUG   ] Got command BAM from MCS: ref #639025
2018-11-27 02:06:03 [INFO    ] Finished processing BAM request
2018-11-27 02:06:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:06:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639025   8 58449  7563030 A  ERROR'
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:06:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:06:03 [DEBUG   ] spawn_process: at 2018-11-27 02:06:03 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:06:05 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 02:06:25 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 02:06:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 02:06:52 [DEBUG   ] Got command BAM from MCS: ref #639027
2018-11-27 02:06:52 [INFO    ] Finished processing BAM request
2018-11-27 02:06:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:06:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639027   8 58449  7612030 A  ERROR'
2018-11-27 02:06:52 [DEBUG   ] Got command BAM from MCS: ref #639026
2018-11-27 02:06:52 [INFO    ] Finished processing BAM request
2018-11-27 02:06:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:06:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639026   8 58449  7612034 A  ERROR'
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:06:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:06:52 [DEBUG   ] spawn_process: at 2018-11-27 02:06:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:06:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:06:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:06:58 [DEBUG   ] Finished updating board flags in 1.382 seconds
2018-11-27 02:07:05 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 02:07:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 02:07:41 [DEBUG   ] Got command BAM from MCS: ref #639033
2018-11-27 02:07:41 [INFO    ] Finished processing BAM request
2018-11-27 02:07:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:07:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639033   8 58449  7661029 A  ERROR'
2018-11-27 02:07:41 [DEBUG   ] Got command BAM from MCS: ref #639032
2018-11-27 02:07:41 [INFO    ] Finished processing BAM request
2018-11-27 02:07:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:07:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639032   8 58449  7661033 A  ERROR'
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:07:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:07:41 [DEBUG   ] spawn_process: at 2018-11-27 02:07:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:07:45 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 02:07:54 [DEBUG   ] Got command RPT from MCS: ref #639034
2018-11-27 02:07:54 [DEBUG   ] summary = ERROR
2018-11-27 02:07:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639034  13 58449  7674461 A  ERRORERROR'
2018-11-27 02:07:54 [DEBUG   ] Got command RPT from MCS: ref #639035
2018-11-27 02:07:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:07:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639035  77 58449  7674665 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:07:54 [DEBUG   ] Got command RPT from MCS: ref #639036
2018-11-27 02:07:54 [DEBUG   ] summary = ERROR
2018-11-27 02:07:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639036  13 58449  7674857 A  ERRORERROR'
2018-11-27 02:07:55 [DEBUG   ] Got command RPT from MCS: ref #639037
2018-11-27 02:07:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:07:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639037  77 58449  7675061 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:07:55 [DEBUG   ] Got command RPT from MCS: ref #639038
2018-11-27 02:07:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:07:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639038  12 58449  7675265 A  ERRORBŸff'
2018-11-27 02:07:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:07:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 02:07:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:07:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:07:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:07:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:07:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:07:58 [DEBUG   ] Finished updating board flags in 1.409 seconds
2018-11-27 02:08:05 [DEBUG   ] Finished updating ant stats in 0.794 seconds
2018-11-27 02:08:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:08:30 [DEBUG   ] Got command BAM from MCS: ref #639040
2018-11-27 02:08:30 [INFO    ] Finished processing BAM request
2018-11-27 02:08:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:08:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639040   8 58449  7710029 A  ERROR'
2018-11-27 02:08:30 [DEBUG   ] Got command BAM from MCS: ref #639039
2018-11-27 02:08:30 [INFO    ] Finished processing BAM request
2018-11-27 02:08:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:08:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639039   8 58449  7710033 A  ERROR'
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:08:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:08:30 [DEBUG   ] spawn_process: at 2018-11-27 02:08:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:08:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 02:08:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:08:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 02:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 02:08:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:08:58 [DEBUG   ] Finished updating board flags in 1.417 seconds
2018-11-27 02:09:06 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2018-11-27 02:09:19 [DEBUG   ] Got command BAM from MCS: ref #639041
2018-11-27 02:09:19 [INFO    ] Finished processing BAM request
2018-11-27 02:09:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:09:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639041   8 58449  7759033 A  ERROR'
2018-11-27 02:09:19 [DEBUG   ] Got command BAM from MCS: ref #639042
2018-11-27 02:09:19 [INFO    ] Finished processing BAM request
2018-11-27 02:09:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:09:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639042   8 58449  7759037 A  ERROR'
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:09:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:09:19 [DEBUG   ] spawn_process: at 2018-11-27 02:09:19 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:09:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:09:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:09:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:09:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:09:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:09:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:09:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:09:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:09:59 [DEBUG   ] Finished updating board flags in 2.207 seconds
2018-11-27 02:10:05 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:10:08 [DEBUG   ] Got command BAM from MCS: ref #639047
2018-11-27 02:10:08 [INFO    ] Finished processing BAM request
2018-11-27 02:10:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:10:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639047   8 58449  7808025 A  ERROR'
2018-11-27 02:10:08 [DEBUG   ] Got command BAM from MCS: ref #639048
2018-11-27 02:10:08 [INFO    ] Finished processing BAM request
2018-11-27 02:10:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:10:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639048   8 58449  7808029 A  ERROR'
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:10:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:10:08 [DEBUG   ] spawn_process: at 2018-11-27 02:10:08 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:10:25 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:10:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 02:10:48 [DEBUG   ] Got command RPT from MCS: ref #639049
2018-11-27 02:10:48 [DEBUG   ] summary = ERROR
2018-11-27 02:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639049  13 58449  7848873 A  ERRORERROR'
2018-11-27 02:10:48 [DEBUG   ] Got command RPT from MCS: ref #639050
2018-11-27 02:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639050  77 58449  7848874 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:10:57 [DEBUG   ] Got command BAM from MCS: ref #639066
2018-11-27 02:10:57 [INFO    ] Finished processing BAM request
2018-11-27 02:10:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:10:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639066   8 58449  7857025 A  ERROR'
2018-11-27 02:10:57 [DEBUG   ] Got command BAM from MCS: ref #639067
2018-11-27 02:10:57 [INFO    ] Finished processing BAM request
2018-11-27 02:10:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:10:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639067   8 58449  7857029 A  ERROR'
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:10:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:10:57 [DEBUG   ] spawn_process: at 2018-11-27 02:10:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:10:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:10:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:10:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:10:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:10:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:10:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:10:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:10:59 [DEBUG   ] Finished updating board flags in 2.201 seconds
2018-11-27 02:11:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:11:25 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 02:11:45 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:11:46 [DEBUG   ] Got command BAM from MCS: ref #639069
2018-11-27 02:11:46 [INFO    ] Finished processing BAM request
2018-11-27 02:11:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:11:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639069   8 58449  7906033 A  ERROR'
2018-11-27 02:11:46 [DEBUG   ] Got command BAM from MCS: ref #639068
2018-11-27 02:11:46 [INFO    ] Finished processing BAM request
2018-11-27 02:11:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:11:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639068   8 58449  7906037 A  ERROR'
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:11:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:11:46 [DEBUG   ] spawn_process: at 2018-11-27 02:11:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:11:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:11:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:11:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:11:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:11:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:11:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:11:59 [DEBUG   ] Finished updating board flags in 2.271 seconds
2018-11-27 02:12:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:12:26 [DEBUG   ] Finished updating ant stats in 1.604 seconds
2018-11-27 02:12:34 [DEBUG   ] Got command BAM from MCS: ref #639070
2018-11-27 02:12:34 [INFO    ] Finished processing BAM request
2018-11-27 02:12:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:12:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639070   8 58449  7954029 A  ERROR'
2018-11-27 02:12:34 [DEBUG   ] Got command BAM from MCS: ref #639071
2018-11-27 02:12:34 [INFO    ] Finished processing BAM request
2018-11-27 02:12:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:12:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639071   8 58449  7954033 A  ERROR'
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 02:12:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 02:12:34 [DEBUG   ] spawn_process: at 2018-11-27 02:12:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 02:12:45 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 02:12:54 [DEBUG   ] Got command RPT from MCS: ref #639074
2018-11-27 02:12:54 [DEBUG   ] summary = ERROR
2018-11-27 02:12:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639074  13 58449  7974541 A  ERRORERROR'
2018-11-27 02:12:54 [DEBUG   ] Got command RPT from MCS: ref #639075
2018-11-27 02:12:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:12:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639075  77 58449  7974745 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:12:54 [DEBUG   ] Got command RPT from MCS: ref #639076
2018-11-27 02:12:54 [DEBUG   ] summary = ERROR
2018-11-27 02:12:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639076  13 58449  7974949 A  ERRORERROR'
2018-11-27 02:12:55 [DEBUG   ] Got command RPT from MCS: ref #639078
2018-11-27 02:12:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:12:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639078  77 58449  7975153 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:12:55 [DEBUG   ] Got command RPT from MCS: ref #639080
2018-11-27 02:12:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:12:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639080  12 58449  7975345 A  ERRORBŸff'
2018-11-27 02:12:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:12:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:12:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:12:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:12:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:12:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:12:59 [DEBUG   ] Finished updating board flags in 2.124 seconds
2018-11-27 02:13:05 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-27 02:13:22 [DEBUG   ] Got command BAM from MCS: ref #639088
2018-11-27 02:13:22 [INFO    ] Finished processing BAM request
2018-11-27 02:13:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:13:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639088   8 58449  8002029 A  ERROR'
2018-11-27 02:13:22 [DEBUG   ] Got command BAM from MCS: ref #639089
2018-11-27 02:13:22 [INFO    ] Finished processing BAM request
2018-11-27 02:13:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:13:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639089   8 58449  8002033 A  ERROR'
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:13:22 [DEBUG   ] spawn_process: at 2018-11-27 02:13:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:13:25 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 02:13:46 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 02:13:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:13:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:13:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 02:13:59 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:13:59 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 02:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:13:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:13:59 [DEBUG   ] Finished updating board flags in 2.001 seconds
2018-11-27 02:14:06 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 02:14:10 [DEBUG   ] Got command BAM from MCS: ref #639090
2018-11-27 02:14:10 [INFO    ] Finished processing BAM request
2018-11-27 02:14:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:14:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639090   8 58449  8050033 A  ERROR'
2018-11-27 02:14:10 [DEBUG   ] Got command BAM from MCS: ref #639091
2018-11-27 02:14:10 [INFO    ] Finished processing BAM request
2018-11-27 02:14:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:14:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639091   8 58449  8050037 A  ERROR'
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:14:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:14:10 [DEBUG   ] spawn_process: at 2018-11-27 02:14:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:14:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 02:14:46 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:14:58 [DEBUG   ] Got command BAM from MCS: ref #639093
2018-11-27 02:14:58 [INFO    ] Finished processing BAM request
2018-11-27 02:14:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:14:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639093   8 58449  8098025 A  ERROR'
2018-11-27 02:14:58 [DEBUG   ] Got command BAM from MCS: ref #639092
2018-11-27 02:14:58 [INFO    ] Finished processing BAM request
2018-11-27 02:14:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:14:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639092   8 58449  8098029 A  ERROR'
2018-11-27 02:14:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 02:14:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 02:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 02:14:58 [DEBUG   ] spawn_process: at 2018-11-27 02:14:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 02:14:59 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:14:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:14:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:14:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:14:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:14:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:14:59 [DEBUG   ] Finished updating board flags in 1.990 seconds
2018-11-27 02:15:06 [DEBUG   ] Finished updating ant stats in 1.192 seconds
2018-11-27 02:15:26 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 02:15:46 [DEBUG   ] Got command BAM from MCS: ref #639094
2018-11-27 02:15:46 [INFO    ] Finished processing BAM request
2018-11-27 02:15:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:15:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639094   8 58449  8146033 A  ERROR'
2018-11-27 02:15:46 [DEBUG   ] Got command BAM from MCS: ref #639095
2018-11-27 02:15:46 [INFO    ] Finished processing BAM request
2018-11-27 02:15:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:15:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639095   8 58449  8146037 A  ERROR'
2018-11-27 02:15:46 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:15:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:15:46 [DEBUG   ] spawn_process: at 2018-11-27 02:15:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:15:48 [DEBUG   ] Got command RPT from MCS: ref #639096
2018-11-27 02:15:48 [DEBUG   ] summary = ERROR
2018-11-27 02:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639096  13 58449  8148881 A  ERRORERROR'
2018-11-27 02:15:48 [DEBUG   ] Got command RPT from MCS: ref #639097
2018-11-27 02:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639097  77 58449  8148882 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:15:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 02:15:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:15:59 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:15:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 02:15:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:15:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:15:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:15:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:15:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:15:59 [DEBUG   ] Finished updating board flags in 1.815 seconds
2018-11-27 02:16:06 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:16:26 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 02:16:34 [DEBUG   ] Got command BAM from MCS: ref #639117
2018-11-27 02:16:34 [INFO    ] Finished processing BAM request
2018-11-27 02:16:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:16:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639117   8 58449  8194025 A  ERROR'
2018-11-27 02:16:34 [DEBUG   ] Got command BAM from MCS: ref #639118
2018-11-27 02:16:34 [INFO    ] Finished processing BAM request
2018-11-27 02:16:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:16:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639118   8 58449  8194029 A  ERROR'
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:16:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:16:34 [DEBUG   ] spawn_process: at 2018-11-27 02:16:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:16:46 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 02:16:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 02:16:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:16:59 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:16:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:16:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:16:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:16:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:16:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:16:59 [DEBUG   ] Finished updating board flags in 1.813 seconds
2018-11-27 02:17:06 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:17:22 [DEBUG   ] Got command BAM from MCS: ref #639120
2018-11-27 02:17:22 [INFO    ] Finished processing BAM request
2018-11-27 02:17:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:17:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639120   8 58449  8242025 A  ERROR'
2018-11-27 02:17:22 [DEBUG   ] Got command BAM from MCS: ref #639119
2018-11-27 02:17:22 [INFO    ] Finished processing BAM request
2018-11-27 02:17:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:17:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639119   8 58449  8242029 A  ERROR'
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:17:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:17:22 [DEBUG   ] spawn_process: at 2018-11-27 02:17:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:17:26 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:17:46 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:17:54 [DEBUG   ] Got command RPT from MCS: ref #639121
2018-11-27 02:17:54 [DEBUG   ] summary = ERROR
2018-11-27 02:17:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639121  13 58449  8274629 A  ERRORERROR'
2018-11-27 02:17:54 [DEBUG   ] Got command RPT from MCS: ref #639122
2018-11-27 02:17:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:17:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639122  77 58449  8274833 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:17:55 [DEBUG   ] Got command RPT from MCS: ref #639123
2018-11-27 02:17:55 [DEBUG   ] summary = ERROR
2018-11-27 02:17:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639123  13 58449  8275037 A  ERRORERROR'
2018-11-27 02:17:55 [DEBUG   ] Got command RPT from MCS: ref #639124
2018-11-27 02:17:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:17:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639124  77 58449  8275229 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:17:55 [DEBUG   ] Got command RPT from MCS: ref #639125
2018-11-27 02:17:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:17:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639125  12 58449  8275433 A  ERRORB ff'
2018-11-27 02:17:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 1
2018-11-27 02:17:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:17:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 02:17:59 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 02:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:17:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:17:59 [DEBUG   ] Finished updating board flags in 1.812 seconds
2018-11-27 02:18:06 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 02:18:10 [DEBUG   ] Got command BAM from MCS: ref #639127
2018-11-27 02:18:10 [INFO    ] Finished processing BAM request
2018-11-27 02:18:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:18:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639127   8 58449  8290029 A  ERROR'
2018-11-27 02:18:10 [DEBUG   ] Got command BAM from MCS: ref #639126
2018-11-27 02:18:10 [INFO    ] Finished processing BAM request
2018-11-27 02:18:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:18:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639126   8 58449  8290033 A  ERROR'
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:18:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:18:10 [DEBUG   ] spawn_process: at 2018-11-27 02:18:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:18:26 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 02:18:46 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-27 02:18:58 [DEBUG   ] Got command BAM from MCS: ref #639129
2018-11-27 02:18:58 [INFO    ] Finished processing BAM request
2018-11-27 02:18:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:18:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639129   8 58449  8338025 A  ERROR'
2018-11-27 02:18:58 [DEBUG   ] Got command BAM from MCS: ref #639128
2018-11-27 02:18:58 [INFO    ] Finished processing BAM request
2018-11-27 02:18:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:18:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639128   8 58449  8338029 A  ERROR'
2018-11-27 02:18:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 02:18:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 02:18:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:18:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:18:58 [DEBUG   ] spawn_process: at 2018-11-27 02:18:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:18:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 02:18:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:18:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2018-11-27 02:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:18:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:18:59 [DEBUG   ] Finished updating board flags in 1.761 seconds
2018-11-27 02:19:07 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2018-11-27 02:19:27 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-27 02:19:46 [DEBUG   ] Got command BAM from MCS: ref #639134
2018-11-27 02:19:46 [INFO    ] Finished processing BAM request
2018-11-27 02:19:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:19:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639134   8 58449  8386029 A  ERROR'
2018-11-27 02:19:46 [DEBUG   ] Got command BAM from MCS: ref #639135
2018-11-27 02:19:46 [INFO    ] Finished processing BAM request
2018-11-27 02:19:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:19:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639135   8 58449  8386033 A  ERROR'
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:19:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:19:46 [DEBUG   ] spawn_process: at 2018-11-27 02:19:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:19:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:19:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 02:19:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:19:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:19:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:19:59 [DEBUG   ] Finished updating board flags in 1.614 seconds
2018-11-27 02:20:07 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:20:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:20:34 [DEBUG   ] Got command BAM from MCS: ref #639136
2018-11-27 02:20:34 [INFO    ] Finished processing BAM request
2018-11-27 02:20:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:20:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639136   8 58449  8434034 A  ERROR'
2018-11-27 02:20:34 [DEBUG   ] Got command BAM from MCS: ref #639137
2018-11-27 02:20:34 [INFO    ] Finished processing BAM request
2018-11-27 02:20:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:20:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639137   8 58449  8434038 A  ERROR'
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 02:20:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 02:20:34 [DEBUG   ] spawn_process: at 2018-11-27 02:20:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 02:20:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:20:48 [DEBUG   ] Got command RPT from MCS: ref #639138
2018-11-27 02:20:48 [DEBUG   ] summary = ERROR
2018-11-27 02:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639138  13 58449  8448881 A  ERRORERROR'
2018-11-27 02:20:48 [DEBUG   ] Got command RPT from MCS: ref #639139
2018-11-27 02:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639139  77 58449  8448882 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:20:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 02:20:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:20:59 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 02:20:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:20:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:20:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:20:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:20:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:20:59 [DEBUG   ] Finished updating board flags in 1.623 seconds
2018-11-27 02:21:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:21:22 [DEBUG   ] Got command BAM from MCS: ref #639156
2018-11-27 02:21:22 [INFO    ] Finished processing BAM request
2018-11-27 02:21:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:21:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639156   8 58449  8482030 A  ERROR'
2018-11-27 02:21:22 [DEBUG   ] Got command BAM from MCS: ref #639155
2018-11-27 02:21:22 [INFO    ] Finished processing BAM request
2018-11-27 02:21:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:21:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639155   8 58449  8482034 A  ERROR'
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:21:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:21:22 [DEBUG   ] spawn_process: at 2018-11-27 02:21:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:21:27 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:21:47 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 02:21:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:21:58 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 02:21:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:21:59 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 02:21:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:21:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:21:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:21:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:21:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:21:59 [DEBUG   ] Finished updating board flags in 1.590 seconds
2018-11-27 02:22:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:22:10 [DEBUG   ] Got command BAM from MCS: ref #639161
2018-11-27 02:22:10 [INFO    ] Finished processing BAM request
2018-11-27 02:22:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:22:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639161   8 58449  8530029 A  ERROR'
2018-11-27 02:22:10 [DEBUG   ] Got command BAM from MCS: ref #639162
2018-11-27 02:22:10 [INFO    ] Finished processing BAM request
2018-11-27 02:22:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:22:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639162   8 58449  8530033 A  ERROR'
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:22:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:22:10 [DEBUG   ] spawn_process: at 2018-11-27 02:22:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:22:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 02:22:47 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:22:54 [DEBUG   ] Got command RPT from MCS: ref #639165
2018-11-27 02:22:54 [DEBUG   ] summary = ERROR
2018-11-27 02:22:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639165  13 58449  8574717 A  ERRORERROR'
2018-11-27 02:22:54 [DEBUG   ] Got command RPT from MCS: ref #639166
2018-11-27 02:22:54 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:22:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639166  77 58449  8574921 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:22:55 [DEBUG   ] Got command RPT from MCS: ref #639167
2018-11-27 02:22:55 [DEBUG   ] summary = ERROR
2018-11-27 02:22:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639167  13 58449  8575113 A  ERRORERROR'
2018-11-27 02:22:55 [DEBUG   ] Got command RPT from MCS: ref #639169
2018-11-27 02:22:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:22:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639169  77 58449  8575317 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:22:55 [DEBUG   ] Got command RPT from MCS: ref #639170
2018-11-27 02:22:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:22:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639170  12 58449  8575521 A  ERRORB¡ff'
2018-11-27 02:22:58 [DEBUG   ] Got command BAM from MCS: ref #639175
2018-11-27 02:22:58 [INFO    ] Finished processing BAM request
2018-11-27 02:22:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:22:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639175   8 58449  8578033 A  ERROR'
2018-11-27 02:22:58 [DEBUG   ] Got command BAM from MCS: ref #639176
2018-11-27 02:22:58 [INFO    ] Finished processing BAM request
2018-11-27 02:22:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:22:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639176   8 58449  8578037 A  ERROR'
2018-11-27 02:22:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:22:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:22:58 [DEBUG   ] spawn_process: at 2018-11-27 02:22:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:22:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:22:59 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 02:22:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2018-11-27 02:22:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:22:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:22:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:22:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:22:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:22:59 [DEBUG   ] Finished updating board flags in 1.455 seconds
2018-11-27 02:23:07 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 02:23:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:23:46 [DEBUG   ] Got command BAM from MCS: ref #639177
2018-11-27 02:23:46 [INFO    ] Finished processing BAM request
2018-11-27 02:23:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:23:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639177   8 58449  8626029 A  ERROR'
2018-11-27 02:23:46 [DEBUG   ] Got command BAM from MCS: ref #639178
2018-11-27 02:23:46 [INFO    ] Finished processing BAM request
2018-11-27 02:23:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:23:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639178   8 58449  8626033 A  ERROR'
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:23:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:23:46 [DEBUG   ] spawn_process: at 2018-11-27 02:23:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:23:47 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 02:23:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:23:59 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:23:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:23:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:23:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:23:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:23:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:23:59 [DEBUG   ] Finished updating board flags in 1.459 seconds
2018-11-27 02:24:07 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:24:27 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 02:24:34 [DEBUG   ] Got command BAM from MCS: ref #639180
2018-11-27 02:24:34 [INFO    ] Finished processing BAM request
2018-11-27 02:24:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:24:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639180   8 58449  8674025 A  ERROR'
2018-11-27 02:24:34 [DEBUG   ] Got command BAM from MCS: ref #639179
2018-11-27 02:24:34 [INFO    ] Finished processing BAM request
2018-11-27 02:24:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:24:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639179   8 58449  8674029 A  ERROR'
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:24:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:24:34 [DEBUG   ] spawn_process: at 2018-11-27 02:24:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:24:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 02:24:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2018-11-27 02:24:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 02:24:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:24:59 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 02:24:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:24:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:24:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:24:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:24:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:24:59 [DEBUG   ] Finished updating board flags in 1.471 seconds
2018-11-27 02:25:07 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 02:25:22 [DEBUG   ] Got command BAM from MCS: ref #639185
2018-11-27 02:25:22 [INFO    ] Finished processing BAM request
2018-11-27 02:25:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:25:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639185   8 58449  8722033 A  ERROR'
2018-11-27 02:25:22 [DEBUG   ] Got command BAM from MCS: ref #639186
2018-11-27 02:25:22 [INFO    ] Finished processing BAM request
2018-11-27 02:25:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:25:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639186   8 58449  8722037 A  ERROR'
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:25:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:25:22 [DEBUG   ] spawn_process: at 2018-11-27 02:25:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:25:27 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 02:25:47 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 02:25:48 [DEBUG   ] Got command RPT from MCS: ref #639187
2018-11-27 02:25:48 [DEBUG   ] summary = ERROR
2018-11-27 02:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639187  13 58449  8748881 A  ERRORERROR'
2018-11-27 02:25:48 [DEBUG   ] Got command RPT from MCS: ref #639188
2018-11-27 02:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639188  77 58449  8748882 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:25:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:25:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:25:59 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 02:25:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:25:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:25:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:25:59 [WARNING ] (13, 1, 'bot')
2018-11-27 02:25:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:25:59 [DEBUG   ] Finished updating board flags in 1.476 seconds
2018-11-27 02:26:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 02:26:10 [DEBUG   ] Got command BAM from MCS: ref #639204
2018-11-27 02:26:10 [INFO    ] Finished processing BAM request
2018-11-27 02:26:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:26:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639204   8 58449  8770033 A  ERROR'
2018-11-27 02:26:10 [DEBUG   ] Got command BAM from MCS: ref #639205
2018-11-27 02:26:10 [INFO    ] Finished processing BAM request
2018-11-27 02:26:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:26:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639205   8 58449  8770037 A  ERROR'
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 02:26:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 02:26:10 [DEBUG   ] spawn_process: at 2018-11-27 02:26:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 02:26:27 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:26:47 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:26:58 [DEBUG   ] Got command BAM from MCS: ref #639207
2018-11-27 02:26:58 [INFO    ] Finished processing BAM request
2018-11-27 02:26:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:26:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639207   8 58449  8818029 A  ERROR'
2018-11-27 02:26:58 [DEBUG   ] Got command BAM from MCS: ref #639206
2018-11-27 02:26:58 [INFO    ] Finished processing BAM request
2018-11-27 02:26:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:26:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639206   8 58449  8818033 A  ERROR'
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:26:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:26:58 [DEBUG   ] spawn_process: at 2018-11-27 02:26:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:26:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:26:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-27 02:27:00 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:27:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:27:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:27:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:27:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:27:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:27:00 [DEBUG   ] Finished updating board flags in 2.200 seconds
2018-11-27 02:27:07 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:27:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 02:27:46 [DEBUG   ] Got command BAM from MCS: ref #639209
2018-11-27 02:27:46 [INFO    ] Finished processing BAM request
2018-11-27 02:27:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:27:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639209   8 58449  8866033 A  ERROR'
2018-11-27 02:27:46 [DEBUG   ] Got command BAM from MCS: ref #639208
2018-11-27 02:27:46 [INFO    ] Finished processing BAM request
2018-11-27 02:27:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:27:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639208   8 58449  8866037 A  ERROR'
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:27:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:27:46 [DEBUG   ] spawn_process: at 2018-11-27 02:27:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:27:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:27:54 [DEBUG   ] Got command RPT from MCS: ref #639210
2018-11-27 02:27:54 [DEBUG   ] summary = ERROR
2018-11-27 02:27:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639210  13 58449  8874805 A  ERRORERROR'
2018-11-27 02:27:55 [DEBUG   ] Got command RPT from MCS: ref #639211
2018-11-27 02:27:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:27:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639211  77 58449  8875009 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:27:55 [DEBUG   ] Got command RPT from MCS: ref #639212
2018-11-27 02:27:55 [DEBUG   ] summary = ERROR
2018-11-27 02:27:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639212  13 58449  8875201 A  ERRORERROR'
2018-11-27 02:27:55 [DEBUG   ] Got command RPT from MCS: ref #639213
2018-11-27 02:27:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:27:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639213  77 58449  8875409 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:27:55 [DEBUG   ] Got command RPT from MCS: ref #639214
2018-11-27 02:27:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:27:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639214  12 58449  8875601 A  ERRORB ff'
2018-11-27 02:27:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 02:27:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:28:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:28:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:28:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:28:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:28:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:28:00 [DEBUG   ] Finished updating board flags in 2.162 seconds
2018-11-27 02:28:07 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:28:27 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:28:34 [DEBUG   ] Got command BAM from MCS: ref #639220
2018-11-27 02:28:34 [INFO    ] Finished processing BAM request
2018-11-27 02:28:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:28:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639220   8 58449  8914025 A  ERROR'
2018-11-27 02:28:34 [DEBUG   ] Got command BAM from MCS: ref #639219
2018-11-27 02:28:34 [INFO    ] Finished processing BAM request
2018-11-27 02:28:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:28:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639219   8 58449  8914029 A  ERROR'
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 02:28:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 02:28:34 [DEBUG   ] spawn_process: at 2018-11-27 02:28:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 02:28:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 02:28:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:28:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 02:28:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:29:00 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:29:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:29:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:29:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:29:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:29:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:29:00 [DEBUG   ] Finished updating board flags in 2.066 seconds
2018-11-27 02:29:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 02:29:22 [DEBUG   ] Got command BAM from MCS: ref #639222
2018-11-27 02:29:22 [INFO    ] Finished processing BAM request
2018-11-27 02:29:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:29:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639222   8 58449  8962029 A  ERROR'
2018-11-27 02:29:22 [DEBUG   ] Got command BAM from MCS: ref #639221
2018-11-27 02:29:22 [INFO    ] Finished processing BAM request
2018-11-27 02:29:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:29:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639221   8 58449  8962033 A  ERROR'
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:29:22 [DEBUG   ] spawn_process: at 2018-11-27 02:29:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:29:27 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:29:47 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 02:29:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:30:00 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:30:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:30:00 [DEBUG   ] Finished updating board flags in 2.024 seconds
2018-11-27 02:30:07 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2018-11-27 02:30:10 [DEBUG   ] Got command BAM from MCS: ref #639223
2018-11-27 02:30:10 [INFO    ] Finished processing BAM request
2018-11-27 02:30:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:30:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639223   8 58449  9010033 A  ERROR'
2018-11-27 02:30:10 [DEBUG   ] Got command BAM from MCS: ref #639224
2018-11-27 02:30:10 [INFO    ] Finished processing BAM request
2018-11-27 02:30:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:30:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639224   8 58449  9010037 A  ERROR'
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 02:30:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 02:30:10 [DEBUG   ] spawn_process: at 2018-11-27 02:30:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 02:30:28 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 02:30:48 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 02:30:48 [DEBUG   ] Got command RPT from MCS: ref #639225
2018-11-27 02:30:48 [DEBUG   ] summary = ERROR
2018-11-27 02:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639225  13 58449  9048873 A  ERRORERROR'
2018-11-27 02:30:48 [DEBUG   ] Got command RPT from MCS: ref #639226
2018-11-27 02:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639226  77 58449  9048874 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:30:58 [DEBUG   ] Got command BAM from MCS: ref #639243
2018-11-27 02:30:58 [INFO    ] Finished processing BAM request
2018-11-27 02:30:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:30:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639243   8 58449  9058033 A  ERROR'
2018-11-27 02:30:58 [DEBUG   ] Got command BAM from MCS: ref #639242
2018-11-27 02:30:58 [INFO    ] Finished processing BAM request
2018-11-27 02:30:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:30:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639242   8 58449  9058037 A  ERROR'
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:30:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:30:58 [DEBUG   ] spawn_process: at 2018-11-27 02:30:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:30:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:31:00 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:31:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:31:00 [DEBUG   ] Finished updating board flags in 1.898 seconds
2018-11-27 02:31:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:31:28 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:31:46 [DEBUG   ] Got command BAM from MCS: ref #639249
2018-11-27 02:31:46 [INFO    ] Finished processing BAM request
2018-11-27 02:31:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:31:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639249   8 58449  9106029 A  ERROR'
2018-11-27 02:31:46 [DEBUG   ] Got command BAM from MCS: ref #639248
2018-11-27 02:31:46 [INFO    ] Finished processing BAM request
2018-11-27 02:31:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:31:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639248   8 58449  9106033 A  ERROR'
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:31:46 [DEBUG   ] spawn_process: at 2018-11-27 02:31:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:31:48 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:31:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:31:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 02:31:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:32:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 02:32:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:32:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:32:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:32:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:32:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:32:00 [DEBUG   ] Finished updating board flags in 1.803 seconds
2018-11-27 02:32:08 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 02:32:28 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2018-11-27 02:32:34 [DEBUG   ] Got command BAM from MCS: ref #639250
2018-11-27 02:32:34 [INFO    ] Finished processing BAM request
2018-11-27 02:32:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:32:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639250   8 58449  9154029 A  ERROR'
2018-11-27 02:32:34 [DEBUG   ] Got command BAM from MCS: ref #639251
2018-11-27 02:32:34 [INFO    ] Finished processing BAM request
2018-11-27 02:32:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:32:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639251   8 58449  9154033 A  ERROR'
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:32:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:32:34 [DEBUG   ] spawn_process: at 2018-11-27 02:32:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:32:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:32:54 [DEBUG   ] Got command RPT from MCS: ref #639254
2018-11-27 02:32:54 [DEBUG   ] summary = ERROR
2018-11-27 02:32:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639254  13 58449  9174885 A  ERRORERROR'
2018-11-27 02:32:55 [DEBUG   ] Got command RPT from MCS: ref #639255
2018-11-27 02:32:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:32:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639255  77 58449  9175089 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:32:55 [DEBUG   ] Got command RPT from MCS: ref #639256
2018-11-27 02:32:55 [DEBUG   ] summary = ERROR
2018-11-27 02:32:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639256  13 58449  9175293 A  ERRORERROR'
2018-11-27 02:32:55 [DEBUG   ] Got command RPT from MCS: ref #639258
2018-11-27 02:32:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:32:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639258  77 58449  9175497 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:32:55 [DEBUG   ] Got command RPT from MCS: ref #639260
2018-11-27 02:32:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:32:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639260  12 58449  9175689 A  ERRORBŸff'
2018-11-27 02:32:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:32:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 02:32:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:33:00 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:33:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:33:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:33:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:33:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:33:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:33:00 [DEBUG   ] Finished updating board flags in 1.832 seconds
2018-11-27 02:33:08 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 02:33:22 [DEBUG   ] Got command BAM from MCS: ref #639265
2018-11-27 02:33:22 [INFO    ] Finished processing BAM request
2018-11-27 02:33:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:33:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639265   8 58449  9202033 A  ERROR'
2018-11-27 02:33:22 [DEBUG   ] Got command BAM from MCS: ref #639264
2018-11-27 02:33:22 [INFO    ] Finished processing BAM request
2018-11-27 02:33:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:33:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639264   8 58449  9202037 A  ERROR'
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:33:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:33:22 [DEBUG   ] spawn_process: at 2018-11-27 02:33:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:33:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 02:33:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 02:33:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:33:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:34:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 02:34:00 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:34:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:34:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:34:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:34:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:34:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:34:00 [DEBUG   ] Finished updating board flags in 1.781 seconds
2018-11-27 02:34:08 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:34:10 [DEBUG   ] Got command BAM from MCS: ref #639271
2018-11-27 02:34:10 [INFO    ] Finished processing BAM request
2018-11-27 02:34:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:34:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639271   8 58449  9250033 A  ERROR'
2018-11-27 02:34:10 [DEBUG   ] Got command BAM from MCS: ref #639270
2018-11-27 02:34:10 [INFO    ] Finished processing BAM request
2018-11-27 02:34:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:34:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639270   8 58449  9250037 A  ERROR'
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:34:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:34:10 [DEBUG   ] spawn_process: at 2018-11-27 02:34:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:34:28 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 02:34:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:34:58 [DEBUG   ] Got command BAM from MCS: ref #639273
2018-11-27 02:34:58 [INFO    ] Finished processing BAM request
2018-11-27 02:34:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:34:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639273   8 58449  9298033 A  ERROR'
2018-11-27 02:34:58 [DEBUG   ] Got command BAM from MCS: ref #639272
2018-11-27 02:34:58 [INFO    ] Finished processing BAM request
2018-11-27 02:34:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:34:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639272   8 58449  9298037 A  ERROR'
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:34:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:34:58 [DEBUG   ] spawn_process: at 2018-11-27 02:34:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:34:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:35:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:35:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:35:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:35:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:35:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:35:00 [DEBUG   ] Finished updating board flags in 1.595 seconds
2018-11-27 02:35:08 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2018-11-27 02:35:28 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 02:35:46 [DEBUG   ] Got command BAM from MCS: ref #639275
2018-11-27 02:35:46 [INFO    ] Finished processing BAM request
2018-11-27 02:35:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:35:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639275   8 58449  9346025 A  ERROR'
2018-11-27 02:35:46 [DEBUG   ] Got command BAM from MCS: ref #639274
2018-11-27 02:35:46 [INFO    ] Finished processing BAM request
2018-11-27 02:35:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:35:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639274   8 58449  9346029 A  ERROR'
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:35:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:35:46 [DEBUG   ] spawn_process: at 2018-11-27 02:35:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:35:48 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:35:48 [DEBUG   ] Got command RPT from MCS: ref #639276
2018-11-27 02:35:48 [DEBUG   ] summary = ERROR
2018-11-27 02:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639276  13 58449  9348877 A  ERRORERROR'
2018-11-27 02:35:48 [DEBUG   ] Got command RPT from MCS: ref #639277
2018-11-27 02:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639277  77 58449  9348878 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:35:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:36:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:36:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:36:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:36:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:36:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:36:00 [DEBUG   ] Finished updating board flags in 1.394 seconds
2018-11-27 02:36:08 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 02:36:29 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-27 02:36:34 [DEBUG   ] Got command BAM from MCS: ref #639294
2018-11-27 02:36:34 [INFO    ] Finished processing BAM request
2018-11-27 02:36:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:36:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639294   8 58449  9394029 A  ERROR'
2018-11-27 02:36:34 [DEBUG   ] Got command BAM from MCS: ref #639293
2018-11-27 02:36:34 [INFO    ] Finished processing BAM request
2018-11-27 02:36:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:36:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639293   8 58449  9394033 A  ERROR'
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:36:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:36:34 [DEBUG   ] spawn_process: at 2018-11-27 02:36:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:36:49 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-27 02:37:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:37:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:37:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:37:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:37:00 [WARNING ] (13, 1, 'bot')
2018-11-27 02:37:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:37:00 [DEBUG   ] Finished updating board flags in 1.361 seconds
2018-11-27 02:37:09 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 02:37:22 [DEBUG   ] Got command BAM from MCS: ref #639300
2018-11-27 02:37:22 [INFO    ] Finished processing BAM request
2018-11-27 02:37:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:37:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639300   8 58449  9442025 A  ERROR'
2018-11-27 02:37:22 [DEBUG   ] Got command BAM from MCS: ref #639299
2018-11-27 02:37:22 [INFO    ] Finished processing BAM request
2018-11-27 02:37:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:37:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639299   8 58449  9442029 A  ERROR'
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:37:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:37:22 [DEBUG   ] spawn_process: at 2018-11-27 02:37:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:37:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 02:37:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 02:37:54 [DEBUG   ] Got command RPT from MCS: ref #639301
2018-11-27 02:37:54 [DEBUG   ] summary = ERROR
2018-11-27 02:37:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639301  13 58449  9474981 A  ERRORERROR'
2018-11-27 02:37:55 [DEBUG   ] Got command RPT from MCS: ref #639302
2018-11-27 02:37:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:37:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639302  77 58449  9475185 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:37:55 [DEBUG   ] Got command RPT from MCS: ref #639303
2018-11-27 02:37:55 [DEBUG   ] summary = ERROR
2018-11-27 02:37:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639303  13 58449  9475377 A  ERRORERROR'
2018-11-27 02:37:55 [DEBUG   ] Got command RPT from MCS: ref #639304
2018-11-27 02:37:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:37:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639304  77 58449  9475581 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:37:55 [DEBUG   ] Got command RPT from MCS: ref #639305
2018-11-27 02:37:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:37:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639305  12 58449  9475785 A  ERRORBŸff'
2018-11-27 02:38:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:38:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:38:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:38:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:38:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:38:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:38:01 [DEBUG   ] Finished updating board flags in 2.066 seconds
2018-11-27 02:38:09 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 02:38:11 [DEBUG   ] Got command BAM from MCS: ref #639307
2018-11-27 02:38:11 [INFO    ] Finished processing BAM request
2018-11-27 02:38:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:38:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639307   8 58449  9491033 A  ERROR'
2018-11-27 02:38:11 [DEBUG   ] Got command BAM from MCS: ref #639306
2018-11-27 02:38:11 [INFO    ] Finished processing BAM request
2018-11-27 02:38:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:38:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639306   8 58449  9491037 A  ERROR'
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 02:38:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 02:38:11 [DEBUG   ] spawn_process: at 2018-11-27 02:38:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 02:38:29 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 02:38:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:39:00 [DEBUG   ] Got command BAM from MCS: ref #639308
2018-11-27 02:39:00 [INFO    ] Finished processing BAM request
2018-11-27 02:39:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:39:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639308   8 58449  9540025 A  ERROR'
2018-11-27 02:39:00 [DEBUG   ] Got command BAM from MCS: ref #639309
2018-11-27 02:39:00 [INFO    ] Finished processing BAM request
2018-11-27 02:39:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:39:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639309   8 58449  9540029 A  ERROR'
2018-11-27 02:39:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 02:39:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:39:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:39:00 [DEBUG   ] spawn_process: at 2018-11-27 02:39:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:39:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:39:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:39:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:39:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:39:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:39:01 [DEBUG   ] Finished updating board flags in 1.976 seconds
2018-11-27 02:39:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 02:39:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 02:39:49 [DEBUG   ] Got command BAM from MCS: ref #639310
2018-11-27 02:39:49 [INFO    ] Finished processing BAM request
2018-11-27 02:39:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:39:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639310   8 58449  9589033 A  ERROR'
2018-11-27 02:39:49 [DEBUG   ] Got command BAM from MCS: ref #639311
2018-11-27 02:39:49 [INFO    ] Finished processing BAM request
2018-11-27 02:39:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:39:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639311   8 58449  9589037 A  ERROR'
2018-11-27 02:39:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:39:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:39:49 [DEBUG   ] spawn_process: at 2018-11-27 02:39:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:40:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:40:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:40:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:40:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:40:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:40:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:40:01 [DEBUG   ] Finished updating board flags in 1.823 seconds
2018-11-27 02:40:09 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:40:29 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 02:40:38 [DEBUG   ] Got command BAM from MCS: ref #639317
2018-11-27 02:40:38 [INFO    ] Finished processing BAM request
2018-11-27 02:40:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:40:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639317   8 58449  9638033 A  ERROR'
2018-11-27 02:40:38 [DEBUG   ] Got command BAM from MCS: ref #639316
2018-11-27 02:40:38 [INFO    ] Finished processing BAM request
2018-11-27 02:40:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:40:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639316   8 58449  9638037 A  ERROR'
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:40:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:40:38 [DEBUG   ] spawn_process: at 2018-11-27 02:40:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:40:48 [DEBUG   ] Got command RPT from MCS: ref #639318
2018-11-27 02:40:48 [DEBUG   ] summary = ERROR
2018-11-27 02:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639318  13 58449  9648881 A  ERRORERROR'
2018-11-27 02:40:48 [DEBUG   ] Got command RPT from MCS: ref #639319
2018-11-27 02:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639319  77 58449  9648882 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:40:49 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:41:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:41:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 02:41:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:41:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 02:41:01 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:41:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:41:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:41:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:41:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:41:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:41:01 [DEBUG   ] Finished updating board flags in 1.822 seconds
2018-11-27 02:41:09 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:41:27 [DEBUG   ] Got command BAM from MCS: ref #639336
2018-11-27 02:41:27 [INFO    ] Finished processing BAM request
2018-11-27 02:41:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:41:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639336   8 58449  9687029 A  ERROR'
2018-11-27 02:41:27 [DEBUG   ] Got command BAM from MCS: ref #639335
2018-11-27 02:41:27 [INFO    ] Finished processing BAM request
2018-11-27 02:41:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:41:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639335   8 58449  9687033 A  ERROR'
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:41:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:41:27 [DEBUG   ] spawn_process: at 2018-11-27 02:41:27 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:41:29 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:41:49 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 02:42:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2018-11-27 02:42:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:42:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:42:01 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 02:42:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2018-11-27 02:42:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:42:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:42:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:42:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:42:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:42:01 [DEBUG   ] Finished updating board flags in 1.896 seconds
2018-11-27 02:42:09 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 02:42:16 [DEBUG   ] Got command BAM from MCS: ref #639337
2018-11-27 02:42:16 [INFO    ] Finished processing BAM request
2018-11-27 02:42:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:42:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639337   8 58449  9736026 A  ERROR'
2018-11-27 02:42:16 [DEBUG   ] Got command BAM from MCS: ref #639338
2018-11-27 02:42:16 [INFO    ] Finished processing BAM request
2018-11-27 02:42:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:42:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639338   8 58449  9736030 A  ERROR'
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:42:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:42:16 [DEBUG   ] spawn_process: at 2018-11-27 02:42:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:42:29 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:42:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:42:55 [DEBUG   ] Got command RPT from MCS: ref #639341
2018-11-27 02:42:55 [DEBUG   ] summary = ERROR
2018-11-27 02:42:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639341  13 58449  9775065 A  ERRORERROR'
2018-11-27 02:42:55 [DEBUG   ] Got command RPT from MCS: ref #639342
2018-11-27 02:42:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:42:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639342  77 58449  9775269 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:42:55 [DEBUG   ] Got command RPT from MCS: ref #639343
2018-11-27 02:42:55 [DEBUG   ] summary = ERROR
2018-11-27 02:42:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639343  13 58449  9775473 A  ERRORERROR'
2018-11-27 02:42:55 [DEBUG   ] Got command RPT from MCS: ref #639345
2018-11-27 02:42:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:42:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639345  77 58449  9775665 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:42:55 [DEBUG   ] Got command RPT from MCS: ref #639346
2018-11-27 02:42:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:42:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639346  12 58449  9775869 A  ERRORB¡ff'
2018-11-27 02:43:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 02:43:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:43:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 02:43:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:43:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:43:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:43:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:43:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:43:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:43:01 [DEBUG   ] Finished updating board flags in 1.695 seconds
2018-11-27 02:43:05 [DEBUG   ] Got command BAM from MCS: ref #639352
2018-11-27 02:43:05 [INFO    ] Finished processing BAM request
2018-11-27 02:43:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:43:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639352   8 58449  9785029 A  ERROR'
2018-11-27 02:43:05 [DEBUG   ] Got command BAM from MCS: ref #639351
2018-11-27 02:43:05 [INFO    ] Finished processing BAM request
2018-11-27 02:43:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:43:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639351   8 58449  9785033 A  ERROR'
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:43:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:43:05 [DEBUG   ] spawn_process: at 2018-11-27 02:43:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:43:09 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:43:29 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:43:49 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 02:43:54 [DEBUG   ] Got command BAM from MCS: ref #639357
2018-11-27 02:43:54 [INFO    ] Finished processing BAM request
2018-11-27 02:43:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:43:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639357   8 58449  9834033 A  ERROR'
2018-11-27 02:43:54 [DEBUG   ] Got command BAM from MCS: ref #639358
2018-11-27 02:43:54 [INFO    ] Finished processing BAM request
2018-11-27 02:43:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:43:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639358   8 58449  9834037 A  ERROR'
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 02:43:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 02:43:54 [DEBUG   ] spawn_process: at 2018-11-27 02:43:54 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 02:44:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:44:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:44:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 02:44:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:44:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:44:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:44:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:44:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:44:01 [DEBUG   ] Finished updating board flags in 1.589 seconds
2018-11-27 02:44:10 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2018-11-27 02:44:30 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:44:43 [DEBUG   ] Got command BAM from MCS: ref #639360
2018-11-27 02:44:43 [INFO    ] Finished processing BAM request
2018-11-27 02:44:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:44:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639360   8 58449  9883033 A  ERROR'
2018-11-27 02:44:43 [DEBUG   ] Got command BAM from MCS: ref #639359
2018-11-27 02:44:43 [INFO    ] Finished processing BAM request
2018-11-27 02:44:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:44:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639359   8 58449  9883037 A  ERROR'
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 02:44:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 02:44:43 [DEBUG   ] spawn_process: at 2018-11-27 02:44:43 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 02:44:50 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:45:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:45:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 02:45:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:45:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 02:45:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 02:45:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 1
2018-11-27 02:45:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:45:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:45:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:45:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:45:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:45:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:45:01 [DEBUG   ] Finished updating board flags in 1.421 seconds
2018-11-27 02:45:10 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:45:30 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:45:32 [DEBUG   ] Got command BAM from MCS: ref #639362
2018-11-27 02:45:32 [INFO    ] Finished processing BAM request
2018-11-27 02:45:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:45:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639362   8 58449  9932025 A  ERROR'
2018-11-27 02:45:32 [DEBUG   ] Got command BAM from MCS: ref #639361
2018-11-27 02:45:32 [INFO    ] Finished processing BAM request
2018-11-27 02:45:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:45:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639361   8 58449  9932029 A  ERROR'
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:45:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:45:32 [DEBUG   ] spawn_process: at 2018-11-27 02:45:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:45:48 [DEBUG   ] Got command RPT from MCS: ref #639363
2018-11-27 02:45:48 [DEBUG   ] summary = ERROR
2018-11-27 02:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639363  13 58449  9948885 A  ERRORERROR'
2018-11-27 02:45:48 [DEBUG   ] Got command RPT from MCS: ref #639364
2018-11-27 02:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639364  77 58449  9948886 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:45:50 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:46:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 02:46:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:46:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 02:46:01 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 02:46:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:46:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:46:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:46:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:46:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:46:01 [DEBUG   ] Finished updating board flags in 1.382 seconds
2018-11-27 02:46:10 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 02:46:21 [DEBUG   ] Got command BAM from MCS: ref #639385
2018-11-27 02:46:21 [INFO    ] Finished processing BAM request
2018-11-27 02:46:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:46:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639385   8 58449  9981033 A  ERROR'
2018-11-27 02:46:21 [DEBUG   ] Got command BAM from MCS: ref #639384
2018-11-27 02:46:21 [INFO    ] Finished processing BAM request
2018-11-27 02:46:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:46:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639384   8 58449  9981037 A  ERROR'
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:46:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:46:21 [DEBUG   ] spawn_process: at 2018-11-27 02:46:21 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:46:30 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 02:46:50 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:47:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:47:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:47:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 02:47:01 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 02:47:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:47:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:47:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:47:01 [WARNING ] (13, 1, 'bot')
2018-11-27 02:47:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:47:01 [DEBUG   ] Finished updating board flags in 1.377 seconds
2018-11-27 02:47:10 [DEBUG   ] Got command BAM from MCS: ref #639387
2018-11-27 02:47:10 [INFO    ] Finished processing BAM request
2018-11-27 02:47:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:47:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639387   8 58449 10030033 A  ERROR'
2018-11-27 02:47:10 [DEBUG   ] Got command BAM from MCS: ref #639386
2018-11-27 02:47:10 [INFO    ] Finished processing BAM request
2018-11-27 02:47:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:47:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639386   8 58449 10030037 A  ERROR'
2018-11-27 02:47:10 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:47:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:47:10 [DEBUG   ] spawn_process: at 2018-11-27 02:47:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:47:30 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 02:47:50 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 02:47:55 [DEBUG   ] Got command RPT from MCS: ref #639388
2018-11-27 02:47:55 [DEBUG   ] summary = ERROR
2018-11-27 02:47:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639388  13 58449 10075153 A  ERRORERROR'
2018-11-27 02:47:55 [DEBUG   ] Got command RPT from MCS: ref #639389
2018-11-27 02:47:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:47:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639389  77 58449 10075357 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:47:55 [DEBUG   ] Got command RPT from MCS: ref #639390
2018-11-27 02:47:55 [DEBUG   ] summary = ERROR
2018-11-27 02:47:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639390  13 58449 10075549 A  ERRORERROR'
2018-11-27 02:47:55 [DEBUG   ] Got command RPT from MCS: ref #639391
2018-11-27 02:47:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:47:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639391  77 58449 10075753 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:47:55 [DEBUG   ] Got command RPT from MCS: ref #639392
2018-11-27 02:47:55 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:47:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639392  12 58449 10075957 A  ERRORB¡ff'
2018-11-27 02:47:59 [DEBUG   ] Got command BAM from MCS: ref #639393
2018-11-27 02:47:59 [INFO    ] Finished processing BAM request
2018-11-27 02:47:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:47:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639393   8 58449 10079033 A  ERROR'
2018-11-27 02:47:59 [DEBUG   ] Got command BAM from MCS: ref #639394
2018-11-27 02:47:59 [INFO    ] Finished processing BAM request
2018-11-27 02:47:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:47:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639394   8 58449 10079037 A  ERROR'
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:47:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:47:59 [DEBUG   ] spawn_process: at 2018-11-27 02:47:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:48:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 02:48:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:48:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 02:48:02 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:48:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:48:02 [DEBUG   ] Finished updating board flags in 2.031 seconds
2018-11-27 02:48:10 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 02:48:30 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 02:48:48 [DEBUG   ] Got command BAM from MCS: ref #639396
2018-11-27 02:48:48 [INFO    ] Finished processing BAM request
2018-11-27 02:48:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:48:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639396   8 58449 10128029 A  ERROR'
2018-11-27 02:48:48 [DEBUG   ] Got command BAM from MCS: ref #639395
2018-11-27 02:48:48 [INFO    ] Finished processing BAM request
2018-11-27 02:48:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:48:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639395   8 58449 10128033 A  ERROR'
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:48:48 [DEBUG   ] spawn_process: at 2018-11-27 02:48:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:48:50 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:49:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:49:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:49:02 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:49:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:49:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:49:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:49:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:49:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:49:02 [DEBUG   ] Finished updating board flags in 1.998 seconds
2018-11-27 02:49:10 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 02:49:30 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 02:49:37 [DEBUG   ] Got command BAM from MCS: ref #639402
2018-11-27 02:49:37 [INFO    ] Finished processing BAM request
2018-11-27 02:49:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:49:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639402   8 58449 10177029 A  ERROR'
2018-11-27 02:49:37 [DEBUG   ] Got command BAM from MCS: ref #639401
2018-11-27 02:49:37 [INFO    ] Finished processing BAM request
2018-11-27 02:49:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:49:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639401   8 58449 10177033 A  ERROR'
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 02:49:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 02:49:37 [DEBUG   ] spawn_process: at 2018-11-27 02:49:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 02:49:50 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 02:50:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:50:02 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 02:50:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:50:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:50:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:50:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:50:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:50:02 [DEBUG   ] Finished updating board flags in 1.810 seconds
2018-11-27 02:50:11 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2018-11-27 02:50:27 [DEBUG   ] Got command BAM from MCS: ref #639404
2018-11-27 02:50:27 [INFO    ] Finished processing BAM request
2018-11-27 02:50:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:50:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639404   8 58449 10227029 A  ERROR'
2018-11-27 02:50:27 [DEBUG   ] Got command BAM from MCS: ref #639403
2018-11-27 02:50:27 [INFO    ] Finished processing BAM request
2018-11-27 02:50:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:50:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639403   8 58449 10227033 A  ERROR'
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:50:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:50:27 [DEBUG   ] spawn_process: at 2018-11-27 02:50:27 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:50:31 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 02:50:48 [DEBUG   ] Got command RPT from MCS: ref #639405
2018-11-27 02:50:48 [DEBUG   ] summary = ERROR
2018-11-27 02:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639405  13 58449 10248885 A  ERRORERROR'
2018-11-27 02:50:48 [DEBUG   ] Got command RPT from MCS: ref #639406
2018-11-27 02:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639406  77 58449 10248886 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:50:51 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 02:51:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:51:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 02:51:02 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:51:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:51:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:51:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:51:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:51:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:51:02 [DEBUG   ] Finished updating board flags in 1.819 seconds
2018-11-27 02:51:11 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:51:17 [DEBUG   ] Got command BAM from MCS: ref #639422
2018-11-27 02:51:17 [INFO    ] Finished processing BAM request
2018-11-27 02:51:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:51:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639422   8 58449 10277033 A  ERROR'
2018-11-27 02:51:17 [DEBUG   ] Got command BAM from MCS: ref #639423
2018-11-27 02:51:17 [INFO    ] Finished processing BAM request
2018-11-27 02:51:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:51:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639423   8 58449 10277037 A  ERROR'
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:51:17 [DEBUG   ] spawn_process: at 2018-11-27 02:51:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:51:31 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:51:51 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:52:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:52:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:52:02 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:52:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:52:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:52:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:52:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:52:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:52:02 [DEBUG   ] Finished updating board flags in 1.786 seconds
2018-11-27 02:52:07 [DEBUG   ] Got command BAM from MCS: ref #639427
2018-11-27 02:52:07 [INFO    ] Finished processing BAM request
2018-11-27 02:52:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:52:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639427   8 58449 10327025 A  ERROR'
2018-11-27 02:52:07 [DEBUG   ] Got command BAM from MCS: ref #639428
2018-11-27 02:52:07 [INFO    ] Finished processing BAM request
2018-11-27 02:52:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:52:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639428   8 58449 10327029 A  ERROR'
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 02:52:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 02:52:07 [DEBUG   ] spawn_process: at 2018-11-27 02:52:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 02:52:11 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:52:31 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:52:51 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 02:52:55 [DEBUG   ] Got command RPT from MCS: ref #639432
2018-11-27 02:52:55 [DEBUG   ] summary = ERROR
2018-11-27 02:52:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639432  13 58449 10375237 A  ERRORERROR'
2018-11-27 02:52:55 [DEBUG   ] Got command RPT from MCS: ref #639433
2018-11-27 02:52:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:52:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639433  77 58449 10375441 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:52:55 [DEBUG   ] Got command RPT from MCS: ref #639434
2018-11-27 02:52:55 [DEBUG   ] summary = ERROR
2018-11-27 02:52:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639434  13 58449 10375633 A  ERRORERROR'
2018-11-27 02:52:55 [DEBUG   ] Got command RPT from MCS: ref #639436
2018-11-27 02:52:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:52:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639436  77 58449 10375837 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:52:56 [DEBUG   ] Got command RPT from MCS: ref #639439
2018-11-27 02:52:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:52:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639439  12 58449 10376041 A  ERRORB ff'
2018-11-27 02:52:57 [DEBUG   ] Got command BAM from MCS: ref #639437
2018-11-27 02:52:57 [INFO    ] Finished processing BAM request
2018-11-27 02:52:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:52:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639437   8 58449 10377029 A  ERROR'
2018-11-27 02:52:57 [DEBUG   ] Got command BAM from MCS: ref #639438
2018-11-27 02:52:57 [INFO    ] Finished processing BAM request
2018-11-27 02:52:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:52:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639438   8 58449 10377033 A  ERROR'
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 02:52:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 02:52:57 [DEBUG   ] spawn_process: at 2018-11-27 02:52:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 02:53:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:53:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 02:53:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:53:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:53:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:53:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:53:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:53:02 [DEBUG   ] Finished updating board flags in 1.601 seconds
2018-11-27 02:53:11 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:53:31 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 02:53:47 [DEBUG   ] Got command BAM from MCS: ref #639445
2018-11-27 02:53:47 [INFO    ] Finished processing BAM request
2018-11-27 02:53:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:53:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639445   8 58449 10427029 A  ERROR'
2018-11-27 02:53:47 [DEBUG   ] Got command BAM from MCS: ref #639444
2018-11-27 02:53:47 [INFO    ] Finished processing BAM request
2018-11-27 02:53:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 02:53:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639444   8 58449 10427033 A  ERROR'
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 02:53:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 02:53:47 [DEBUG   ] spawn_process: at 2018-11-27 02:53:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 02:53:51 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:54:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:54:02 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 02:54:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:54:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:54:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:54:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:54:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:54:02 [DEBUG   ] Finished updating board flags in 1.659 seconds
2018-11-27 02:54:11 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 02:54:31 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 02:54:33 [DEBUG   ] Got command STP from MCS: ref #639447
2018-11-27 02:54:33 [INFO    ] Finished processing BAM request
2018-11-27 02:54:33 [INFO    ] Shutdown the adder chain on beam #4
2018-11-27 02:54:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   639447   8 58449 10473025 A  ERROR'
2018-11-27 02:54:33 [DEBUG   ] Got command STP from MCS: ref #639446
2018-11-27 02:54:33 [INFO    ] Finished processing BAM request
2018-11-27 02:54:33 [INFO    ] Shutdown the adder chain on beam #3
2018-11-27 02:54:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   639446   8 58449 10473029 A  ERROR'
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 02:54:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 02:54:33 [DEBUG   ] spawn_process: at 2018-11-27 02:54:33 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 02:54:51 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2018-11-27 02:55:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:55:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:55:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:55:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:55:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:55:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:55:02 [DEBUG   ] Finished updating board flags in 1.543 seconds
2018-11-27 02:55:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 02:55:31 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 02:55:48 [DEBUG   ] Got command RPT from MCS: ref #639452
2018-11-27 02:55:48 [DEBUG   ] summary = ERROR
2018-11-27 02:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639452  13 58449 10548881 A  ERRORERROR'
2018-11-27 02:55:48 [DEBUG   ] Got command RPT from MCS: ref #639453
2018-11-27 02:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639453  77 58449 10548882 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:55:51 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 02:56:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 02:56:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:56:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:56:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:56:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:56:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:56:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:56:02 [DEBUG   ] Finished updating board flags in 1.454 seconds
2018-11-27 02:56:11 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 02:56:31 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 02:56:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 02:57:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:57:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:57:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:57:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:57:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:57:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:57:02 [DEBUG   ] Finished updating board flags in 1.534 seconds
2018-11-27 02:57:11 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 02:57:31 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 02:57:51 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:57:55 [DEBUG   ] Got command RPT from MCS: ref #639469
2018-11-27 02:57:55 [DEBUG   ] summary = ERROR
2018-11-27 02:57:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639469  13 58449 10675329 A  ERRORERROR'
2018-11-27 02:57:55 [DEBUG   ] Got command RPT from MCS: ref #639470
2018-11-27 02:57:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:57:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639470  77 58449 10675521 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:57:55 [DEBUG   ] Got command RPT from MCS: ref #639471
2018-11-27 02:57:55 [DEBUG   ] summary = ERROR
2018-11-27 02:57:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639471  13 58449 10675725 A  ERRORERROR'
2018-11-27 02:57:55 [DEBUG   ] Got command RPT from MCS: ref #639472
2018-11-27 02:57:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 02:57:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639472  77 58449 10675929 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 02:57:56 [DEBUG   ] Got command RPT from MCS: ref #639473
2018-11-27 02:57:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 02:57:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639473  12 58449 10676133 A  ERRORB˜™š'
2018-11-27 02:58:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:58:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:58:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:58:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:58:02 [WARNING ] (13, 1, 'bot')
2018-11-27 02:58:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:58:02 [DEBUG   ] Finished updating board flags in 1.527 seconds
2018-11-27 02:58:11 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 02:58:31 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 02:58:51 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 02:59:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 02:59:03 [WARNING ] (13, 1, 'bot')
2018-11-27 02:59:03 [WARNING ] (13, 1, 'bot')
2018-11-27 02:59:03 [WARNING ] (13, 1, 'bot')
2018-11-27 02:59:03 [WARNING ] (13, 1, 'bot')
2018-11-27 02:59:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 02:59:03 [DEBUG   ] Finished updating board flags in 2.161 seconds
2018-11-27 02:59:11 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:59:31 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 02:59:51 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 03:00:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:00:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-27 03:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:00:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:00:03 [DEBUG   ] Finished updating board flags in 1.946 seconds
2018-11-27 03:00:11 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 03:00:31 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:00:48 [DEBUG   ] Got command RPT from MCS: ref #639478
2018-11-27 03:00:48 [DEBUG   ] summary = ERROR
2018-11-27 03:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639478  13 58449 10848885 A  ERRORERROR'
2018-11-27 03:00:48 [DEBUG   ] Got command RPT from MCS: ref #639479
2018-11-27 03:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639479  77 58449 10848886 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:00:52 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 03:01:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:01:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:01:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:01:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:01:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:01:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:01:03 [DEBUG   ] Finished updating board flags in 1.805 seconds
2018-11-27 03:01:12 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 03:01:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 03:01:52 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 03:02:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:02:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:02:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:02:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:02:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:02:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:02:03 [DEBUG   ] Finished updating board flags in 1.775 seconds
2018-11-27 03:02:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 03:02:32 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 03:02:52 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:02:55 [DEBUG   ] Got command RPT from MCS: ref #639501
2018-11-27 03:02:55 [DEBUG   ] summary = ERROR
2018-11-27 03:02:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639501  13 58449 10975405 A  ERRORERROR'
2018-11-27 03:02:55 [DEBUG   ] Got command RPT from MCS: ref #639502
2018-11-27 03:02:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:02:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639502  77 58449 10975609 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:02:55 [DEBUG   ] Got command RPT from MCS: ref #639503
2018-11-27 03:02:55 [DEBUG   ] summary = ERROR
2018-11-27 03:02:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639503  13 58449 10975813 A  ERRORERROR'
2018-11-27 03:02:56 [DEBUG   ] Got command RPT from MCS: ref #639504
2018-11-27 03:02:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:02:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639504  77 58449 10976005 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:02:56 [DEBUG   ] Got command RPT from MCS: ref #639506
2018-11-27 03:02:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:02:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639506  12 58449 10976209 A  ERRORB•™š'
2018-11-27 03:03:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:03:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:03:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:03:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:03:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:03:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:03:03 [DEBUG   ] Finished updating board flags in 1.659 seconds
2018-11-27 03:03:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 03:03:32 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 03:03:52 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 03:04:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:04:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 03:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:04:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:04:03 [DEBUG   ] Finished updating board flags in 1.543 seconds
2018-11-27 03:04:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 03:04:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 03:04:52 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 03:05:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:05:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:05:03 [DEBUG   ] Finished updating board flags in 1.423 seconds
2018-11-27 03:05:12 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 03:05:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 03:05:48 [DEBUG   ] Got command RPT from MCS: ref #639515
2018-11-27 03:05:48 [DEBUG   ] summary = ERROR
2018-11-27 03:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639515  13 58449 11148889 A  ERRORERROR'
2018-11-27 03:05:48 [DEBUG   ] Got command RPT from MCS: ref #639516
2018-11-27 03:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639516  77 58449 11148890 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:05:52 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 03:06:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:06:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:06:03 [DEBUG   ] Finished updating board flags in 1.453 seconds
2018-11-27 03:06:12 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 03:06:32 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 03:06:52 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 03:07:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 03:07:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:07:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:07:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:07:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:07:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:07:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:07:03 [DEBUG   ] Finished updating board flags in 1.456 seconds
2018-11-27 03:07:12 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:07:32 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 03:07:52 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 03:07:55 [DEBUG   ] Got command RPT from MCS: ref #639536
2018-11-27 03:07:55 [DEBUG   ] summary = ERROR
2018-11-27 03:07:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639536  13 58449 11275489 A  ERRORERROR'
2018-11-27 03:07:55 [DEBUG   ] Got command RPT from MCS: ref #639537
2018-11-27 03:07:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:07:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639537  77 58449 11275693 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:07:55 [DEBUG   ] Got command RPT from MCS: ref #639538
2018-11-27 03:07:55 [DEBUG   ] summary = ERROR
2018-11-27 03:07:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639538  13 58449 11275897 A  ERRORERROR'
2018-11-27 03:07:56 [DEBUG   ] Got command RPT from MCS: ref #639539
2018-11-27 03:07:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:07:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639539  77 58449 11276101 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:07:56 [DEBUG   ] Got command RPT from MCS: ref #639540
2018-11-27 03:07:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:07:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639540  12 58449 11276293 A  ERRORB—™š'
2018-11-27 03:08:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 03:08:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:08:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:08:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:08:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:08:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:08:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:08:03 [DEBUG   ] Finished updating board flags in 1.449 seconds
2018-11-27 03:08:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 03:08:32 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:08:52 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 03:09:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:09:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:09:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:09:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:09:03 [WARNING ] (13, 1, 'bot')
2018-11-27 03:09:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:09:03 [DEBUG   ] Finished updating board flags in 1.544 seconds
2018-11-27 03:09:12 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 03:09:32 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:09:53 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 03:10:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:10:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:10:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:10:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:10:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:10:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:10:04 [DEBUG   ] Finished updating board flags in 2.168 seconds
2018-11-27 03:10:13 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 03:10:33 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:10:48 [DEBUG   ] Got command RPT from MCS: ref #639545
2018-11-27 03:10:48 [DEBUG   ] summary = ERROR
2018-11-27 03:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639545  13 58449 11448889 A  ERRORERROR'
2018-11-27 03:10:48 [DEBUG   ] Got command RPT from MCS: ref #639546
2018-11-27 03:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639546  77 58449 11448890 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:10:53 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 03:11:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:11:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:11:04 [DEBUG   ] Finished updating board flags in 1.927 seconds
2018-11-27 03:11:13 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 03:11:33 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:11:53 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 03:12:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:12:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:12:04 [DEBUG   ] Finished updating board flags in 2.060 seconds
2018-11-27 03:12:13 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 03:12:33 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 03:12:53 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 03:12:55 [DEBUG   ] Got command RPT from MCS: ref #639564
2018-11-27 03:12:55 [DEBUG   ] summary = ERROR
2018-11-27 03:12:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639564  13 58449 11575581 A  ERRORERROR'
2018-11-27 03:12:55 [DEBUG   ] Got command RPT from MCS: ref #639565
2018-11-27 03:12:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:12:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639565  77 58449 11575785 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:12:55 [DEBUG   ] Got command RPT from MCS: ref #639566
2018-11-27 03:12:55 [DEBUG   ] summary = ERROR
2018-11-27 03:12:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639566  13 58449 11575977 A  ERRORERROR'
2018-11-27 03:12:56 [DEBUG   ] Got command RPT from MCS: ref #639567
2018-11-27 03:12:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:12:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639567  77 58449 11576181 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:12:56 [DEBUG   ] Got command RPT from MCS: ref #639569
2018-11-27 03:12:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:12:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639569  12 58449 11576385 A  ERRORB–™š'
2018-11-27 03:13:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:13:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:13:04 [DEBUG   ] Finished updating board flags in 1.749 seconds
2018-11-27 03:13:13 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 03:13:33 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2018-11-27 03:13:53 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 03:14:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:14:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:14:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:14:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:14:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:14:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:14:04 [DEBUG   ] Finished updating board flags in 1.632 seconds
2018-11-27 03:14:13 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 03:14:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:14:53 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 03:15:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:15:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-27 03:15:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:15:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:15:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:15:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:15:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:15:04 [DEBUG   ] Finished updating board flags in 1.571 seconds
2018-11-27 03:15:13 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 03:15:34 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2018-11-27 03:15:48 [DEBUG   ] Got command RPT from MCS: ref #639578
2018-11-27 03:15:48 [DEBUG   ] summary = ERROR
2018-11-27 03:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639578  13 58449 11748885 A  ERRORERROR'
2018-11-27 03:15:48 [DEBUG   ] Got command RPT from MCS: ref #639579
2018-11-27 03:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639579  77 58449 11748886 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:15:54 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-27 03:16:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:16:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:16:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:16:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:16:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:16:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:16:04 [DEBUG   ] Finished updating board flags in 1.406 seconds
2018-11-27 03:16:14 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 03:16:34 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 03:16:54 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 03:17:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:17:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:17:04 [DEBUG   ] Finished updating board flags in 1.415 seconds
2018-11-27 03:17:14 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 03:17:34 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 03:17:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 03:17:55 [DEBUG   ] Got command RPT from MCS: ref #639599
2018-11-27 03:17:55 [DEBUG   ] summary = ERROR
2018-11-27 03:17:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639599  13 58449 11875665 A  ERRORERROR'
2018-11-27 03:17:55 [DEBUG   ] Got command RPT from MCS: ref #639600
2018-11-27 03:17:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:17:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639600  77 58449 11875869 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:17:56 [DEBUG   ] Got command RPT from MCS: ref #639601
2018-11-27 03:17:56 [DEBUG   ] summary = ERROR
2018-11-27 03:17:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639601  13 58449 11876073 A  ERRORERROR'
2018-11-27 03:17:56 [DEBUG   ] Got command RPT from MCS: ref #639602
2018-11-27 03:17:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:17:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639602  77 58449 11876277 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:17:56 [DEBUG   ] Got command RPT from MCS: ref #639603
2018-11-27 03:17:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:17:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639603  12 58449 11876469 A  ERRORB”™š'
2018-11-27 03:18:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:18:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:18:04 [DEBUG   ] Finished updating board flags in 1.390 seconds
2018-11-27 03:18:14 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 03:18:34 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 03:18:54 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 03:19:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 03:19:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:19:04 [DEBUG   ] Finished updating board flags in 1.465 seconds
2018-11-27 03:19:14 [DEBUG   ] Finished updating ant stats in 0.794 seconds
2018-11-27 03:19:34 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 03:19:54 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 03:20:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:20:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:20:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:20:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:20:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:20:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:20:05 [DEBUG   ] Finished updating board flags in 2.215 seconds
2018-11-27 03:20:12 [DEBUG   ] Got command TBW from MCS: ref #639609
2018-11-27 03:20:12 [DEBUG   ] tbw = exited with status True, code 0x00
2018-11-27 03:20:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBW   639609   8 58449 12012049 A  ERROR'
2018-11-27 03:20:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 pkill tbn
2018-11-27 03:20:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 pkill tbn
2018-11-27 03:20:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 pkill tbn
2018-11-27 03:20:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 pkill tbn
2018-11-27 03:20:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 pkill tbn
2018-11-27 03:20:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 pkill tbn
2018-11-27 03:20:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 pkill tbn
2018-11-27 03:20:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 pkill tbn
2018-11-27 03:20:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 pkill tbn
2018-11-27 03:20:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 pkill tbn
2018-11-27 03:20:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 pkill tbn
2018-11-27 03:20:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 pkill tbn
2018-11-27 03:20:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 pkill tbn
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa27', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '1', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa24', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '2', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa23', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '3', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa18', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '4', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa10', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '5', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa32', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '6', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa11', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '7', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa30', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '8', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa12', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '9', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa8', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '10', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa13', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '11', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa26', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '12', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa14', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '13', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa22', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '14', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa20', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '15', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa1', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '16', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa5', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '17', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa25', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '18', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa35', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '19', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa19', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '20', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa21', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '21', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa6', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '22', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa7', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '23', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa33', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '24', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa15', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '25', '12000000']
2018-11-27 03:20:15 [DEBUG   ] spawn_process: at 2018-11-27 03:20:15 running ['/usr/local/bin/lwa28', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '26', '12000000']
2018-11-27 03:20:15 [INFO    ] Finished spawning TBW
2018-11-27 03:20:48 [DEBUG   ] Got command RPT from MCS: ref #639612
2018-11-27 03:20:48 [DEBUG   ] summary = ERROR
2018-11-27 03:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639612  13 58449 12048885 A  ERRORERROR'
2018-11-27 03:20:48 [DEBUG   ] Got command RPT from MCS: ref #639613
2018-11-27 03:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639613  77 58449 12048886 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:21:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:21:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:21:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:21:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:21:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:21:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:21:05 [DEBUG   ] Finished updating board flags in 2.004 seconds
2018-11-27 03:22:03 [INFO    ] __waitForTBW - TBW finished in 108.229 s
2018-11-27 03:22:04 [DEBUG   ] Finished updating ant stats in 0.502 seconds
2018-11-27 03:22:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:22:04 [INFO    ] Started antennas statistics background thread
2018-11-27 03:22:04 [INFO    ] Resuming TBN at 38.100 MHz after TBW acquisition
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '1', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '2', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '3', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '4', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '5', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '6', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '7', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '8', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '9', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '10', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '11', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '12', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '13', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '14', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '15', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '16', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '17', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '18', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '19', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '20', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '21', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '22', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '23', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '24', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '25', '1543288926']
2018-11-27 03:22:04 [DEBUG   ] spawn_process: at 2018-11-27 03:22:04 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '26', '1543288926']
2018-11-27 03:22:04 [INFO    ] Finished spawning TBN in 0.050 s
2018-11-27 03:22:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:22:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:22:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:22:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:22:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:22:05 [DEBUG   ] Finished updating board flags in 1.937 seconds
2018-11-27 03:22:24 [DEBUG   ] Finished updating ant stats in 0.438 seconds
2018-11-27 03:22:44 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 03:22:55 [DEBUG   ] Got command RPT from MCS: ref #639640
2018-11-27 03:22:55 [DEBUG   ] summary = ERROR
2018-11-27 03:22:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639640  13 58449 12175753 A  ERRORERROR'
2018-11-27 03:22:55 [DEBUG   ] Got command RPT from MCS: ref #639641
2018-11-27 03:22:55 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:22:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639641  77 58449 12175957 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:22:56 [DEBUG   ] Got command RPT from MCS: ref #639642
2018-11-27 03:22:56 [DEBUG   ] summary = ERROR
2018-11-27 03:22:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639642  13 58449 12176161 A  ERRORERROR'
2018-11-27 03:22:56 [DEBUG   ] Got command RPT from MCS: ref #639644
2018-11-27 03:22:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:22:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639644  77 58449 12176365 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:22:56 [DEBUG   ] Got command RPT from MCS: ref #639645
2018-11-27 03:22:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:22:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639645  12 58449 12176557 A  ERRORB—™š'
2018-11-27 03:23:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:23:05 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 03:23:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:23:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:23:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:23:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:23:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:23:05 [DEBUG   ] Finished updating board flags in 2.003 seconds
2018-11-27 03:23:25 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 03:23:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:24:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:24:05 [DEBUG   ] Finished updating ant stats in 1.248 seconds
2018-11-27 03:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:24:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:24:05 [DEBUG   ] Finished updating board flags in 2.055 seconds
2018-11-27 03:24:25 [DEBUG   ] Finished updating ant stats in 1.155 seconds
2018-11-27 03:24:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 03:25:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:25:05 [DEBUG   ] Finished updating ant stats in 1.038 seconds
2018-11-27 03:25:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:25:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:25:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:25:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:25:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:25:05 [DEBUG   ] Finished updating board flags in 1.995 seconds
2018-11-27 03:25:25 [DEBUG   ] Finished updating ant stats in 0.960 seconds
2018-11-27 03:25:45 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 03:25:48 [DEBUG   ] Got command RPT from MCS: ref #639654
2018-11-27 03:25:48 [DEBUG   ] summary = ERROR
2018-11-27 03:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639654  13 58449 12348897 A  ERRORERROR'
2018-11-27 03:25:48 [DEBUG   ] Got command RPT from MCS: ref #639655
2018-11-27 03:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639655  77 58449 12348898 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:26:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:26:05 [DEBUG   ] Finished updating ant stats in 0.646 seconds
2018-11-27 03:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:26:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:26:05 [DEBUG   ] Finished updating board flags in 1.788 seconds
2018-11-27 03:26:25 [DEBUG   ] Finished updating ant stats in 0.558 seconds
2018-11-27 03:26:45 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-27 03:27:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 03:27:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:27:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:27:05 [DEBUG   ] Finished updating board flags in 1.625 seconds
2018-11-27 03:27:06 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-27 03:27:26 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 03:27:46 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 03:27:55 [DEBUG   ] Got command RPT from MCS: ref #639671
2018-11-27 03:27:55 [DEBUG   ] summary = ERROR
2018-11-27 03:27:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639671  13 58449 12475849 A  ERRORERROR'
2018-11-27 03:27:56 [DEBUG   ] Got command RPT from MCS: ref #639672
2018-11-27 03:27:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:27:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639672  77 58449 12476041 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:27:56 [DEBUG   ] Got command RPT from MCS: ref #639673
2018-11-27 03:27:56 [DEBUG   ] summary = ERROR
2018-11-27 03:27:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639673  13 58449 12476245 A  ERRORERROR'
2018-11-27 03:27:56 [DEBUG   ] Got command RPT from MCS: ref #639674
2018-11-27 03:27:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:27:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639674  77 58449 12476449 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:27:56 [DEBUG   ] Got command RPT from MCS: ref #639675
2018-11-27 03:27:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:27:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639675  12 58449 12476653 A  ERRORB–™š'
2018-11-27 03:28:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:28:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:28:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:28:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:28:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:28:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:28:05 [DEBUG   ] Finished updating board flags in 1.563 seconds
2018-11-27 03:28:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 03:28:26 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:28:46 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 03:29:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:29:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:29:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:29:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:29:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:29:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:29:05 [DEBUG   ] Finished updating board flags in 1.520 seconds
2018-11-27 03:29:06 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 03:29:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 03:29:46 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:30:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:30:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:30:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:30:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:30:05 [WARNING ] (13, 1, 'bot')
2018-11-27 03:30:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:30:05 [DEBUG   ] Finished updating board flags in 1.541 seconds
2018-11-27 03:30:06 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 03:30:26 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 03:30:46 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 03:30:48 [DEBUG   ] Got command RPT from MCS: ref #639680
2018-11-27 03:30:48 [DEBUG   ] summary = ERROR
2018-11-27 03:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639680  13 58449 12648893 A  ERRORERROR'
2018-11-27 03:30:48 [DEBUG   ] Got command RPT from MCS: ref #639681
2018-11-27 03:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639681  77 58449 12648894 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:31:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:31:06 [DEBUG   ] Finished updating ant stats in 1.050 seconds
2018-11-27 03:31:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:31:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:31:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:31:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:31:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:31:06 [DEBUG   ] Finished updating board flags in 2.238 seconds
2018-11-27 03:31:26 [DEBUG   ] Finished updating ant stats in 0.951 seconds
2018-11-27 03:31:46 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 03:32:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:32:06 [DEBUG   ] Finished updating ant stats in 0.843 seconds
2018-11-27 03:32:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:32:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:32:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:32:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:32:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:32:06 [DEBUG   ] Finished updating board flags in 2.120 seconds
2018-11-27 03:32:26 [DEBUG   ] Finished updating ant stats in 0.758 seconds
2018-11-27 03:32:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 03:32:55 [DEBUG   ] Got command RPT from MCS: ref #639703
2018-11-27 03:32:55 [DEBUG   ] summary = ERROR
2018-11-27 03:32:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639703  13 58449 12775937 A  ERRORERROR'
2018-11-27 03:32:56 [DEBUG   ] Got command RPT from MCS: ref #639704
2018-11-27 03:32:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:32:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639704  77 58449 12776129 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:32:56 [DEBUG   ] Got command RPT from MCS: ref #639705
2018-11-27 03:32:56 [DEBUG   ] summary = ERROR
2018-11-27 03:32:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639705  13 58449 12776333 A  ERRORERROR'
2018-11-27 03:32:56 [DEBUG   ] Got command RPT from MCS: ref #639707
2018-11-27 03:32:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:32:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639707  77 58449 12776537 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:32:56 [DEBUG   ] Got command RPT from MCS: ref #639709
2018-11-27 03:32:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:32:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639709  12 58449 12776741 A  ERRORB•™š'
2018-11-27 03:33:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:33:06 [DEBUG   ] Finished updating ant stats in 0.645 seconds
2018-11-27 03:33:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:33:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:33:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:33:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:33:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:33:06 [DEBUG   ] Finished updating board flags in 2.010 seconds
2018-11-27 03:33:26 [DEBUG   ] Finished updating ant stats in 0.554 seconds
2018-11-27 03:33:47 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 03:34:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:34:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:34:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:34:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:34:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:34:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:34:06 [DEBUG   ] Finished updating board flags in 2.048 seconds
2018-11-27 03:34:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 03:34:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:34:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:35:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:35:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:35:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:35:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:35:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:35:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:35:06 [DEBUG   ] Finished updating board flags in 1.951 seconds
2018-11-27 03:35:07 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 03:35:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:35:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 03:35:48 [DEBUG   ] Got command RPT from MCS: ref #639717
2018-11-27 03:35:48 [DEBUG   ] summary = ERROR
2018-11-27 03:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639717  13 58449 12948889 A  ERRORERROR'
2018-11-27 03:35:48 [DEBUG   ] Got command RPT from MCS: ref #639718
2018-11-27 03:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639718  77 58449 12948890 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:36:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 03:36:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:36:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:36:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:36:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:36:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:36:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:36:06 [DEBUG   ] Finished updating board flags in 1.828 seconds
2018-11-27 03:36:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:36:27 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 03:36:47 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 03:37:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:37:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:37:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:37:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:37:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:37:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:37:06 [DEBUG   ] Finished updating board flags in 1.774 seconds
2018-11-27 03:37:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 03:37:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:37:47 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 03:37:56 [DEBUG   ] Got command RPT from MCS: ref #639738
2018-11-27 03:37:56 [DEBUG   ] summary = ERROR
2018-11-27 03:37:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639738  13 58449 13076017 A  ERRORERROR'
2018-11-27 03:37:56 [DEBUG   ] Got command RPT from MCS: ref #639739
2018-11-27 03:37:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:37:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639739  77 58449 13076221 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:37:56 [DEBUG   ] Got command RPT from MCS: ref #639740
2018-11-27 03:37:56 [DEBUG   ] summary = ERROR
2018-11-27 03:37:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639740  13 58449 13076413 A  ERRORERROR'
2018-11-27 03:37:56 [DEBUG   ] Got command RPT from MCS: ref #639741
2018-11-27 03:37:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:37:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639741  77 58449 13076617 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:37:56 [DEBUG   ] Got command RPT from MCS: ref #639742
2018-11-27 03:37:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:37:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639742  12 58449 13076821 A  ERRORB–™š'
2018-11-27 03:38:00 [DEBUG   ] Got command RPT from MCS: ref #639743
2018-11-27 03:38:00 [DEBUG   ] summary = ERROR
2018-11-27 03:38:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639743  13 58449 13080421 A  ERRORERROR'
2018-11-27 03:38:00 [DEBUG   ] Got command RPT from MCS: ref #639744
2018-11-27 03:38:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:38:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639744  77 58449 13080422 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:38:05 [DEBUG   ] Got command INI from MCS: ref #639745
2018-11-27 03:38:05 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2018-11-27 03:38:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   639745   8 58449 13085429 ABOOTING'
2018-11-27 03:38:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:38:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-27 03:38:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:38:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:38:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:38:06 [WARNING ] (13, 1, 'bot')
2018-11-27 03:38:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:38:06 [DEBUG   ] Finished updating board flags in 1.635 seconds
2018-11-27 03:38:07 [DEBUG   ] Finished updating ant stats in 1.243 seconds
2018-11-27 03:38:29 [INFO    ] -------------------
2018-11-27 03:38:29 [INFO    ]  LWA board status
2018-11-27 03:38:29 [INFO    ] -------------------
2018-11-27 03:38:29 [INFO    ] DP 1: 2 lwa27
2018-11-27 03:38:29 [INFO    ] DP 1: 3 lwa24
2018-11-27 03:38:29 [INFO    ] DP 1: 4 lwa23
2018-11-27 03:38:29 [INFO    ] DP 1: 5 lwa18
2018-11-27 03:38:29 [INFO    ] DP 1: 6 lwa10
2018-11-27 03:38:29 [INFO    ] DP 1: 7 lwa32
2018-11-27 03:38:29 [INFO    ] DP 1: 8 lwa11
2018-11-27 03:38:29 [INFO    ] DP 1: 9 lwa30
2018-11-27 03:38:29 [INFO    ] DP 1: 10 lwa12
2018-11-27 03:38:29 [INFO    ] DP 1: 11 lwa8
2018-11-27 03:38:29 [INFO    ] DP 1: 12 lwa13
2018-11-27 03:38:29 [INFO    ] DP 1: 13 lwa26
2018-11-27 03:38:29 [INFO    ] DP 1: 14 lwa14
2018-11-27 03:38:29 [INFO    ] DP 1: 16 lwa22
2018-11-27 03:38:29 [INFO    ] DP 1: 17 lwa20
2018-11-27 03:38:29 [INFO    ] DP 1: 18 lwa1
2018-11-27 03:38:29 [INFO    ] DP 1: 19 lwa5
2018-11-27 03:38:29 [INFO    ] DP 1: 20 lwa25
2018-11-27 03:38:29 [INFO    ] DP 1: 21 lwa35
2018-11-27 03:38:29 [INFO    ] DP 1: 22 lwa19
2018-11-27 03:38:29 [INFO    ] DP 1: 23 lwa21
2018-11-27 03:38:29 [INFO    ] DP 1: 24 lwa6
2018-11-27 03:38:29 [INFO    ] DP 1: 25 lwa7
2018-11-27 03:38:29 [INFO    ] DP 1: 26 lwa33
2018-11-27 03:38:29 [INFO    ] DP 1: 27 lwa15
2018-11-27 03:38:29 [INFO    ] DP 1: 28 lwa28
2018-11-27 03:38:29 [INFO    ] DP 2: 1 lwa16
2018-11-27 03:38:29 [INFO    ] DP 2: 15 lwa34
2018-11-27 03:38:29 [INFO    ] Total of 28 boards
2018-11-27 03:38:29 [INFO    ] Total of 260 stands
2018-11-27 03:38:29 [INFO    ] -------------------
2018-11-27 03:38:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14857
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14858
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14859
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14860
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14861
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14862
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14863
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14864
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14865
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14866
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14867
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14868
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14869
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14870
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14871
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14872
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14873
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14874
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14875
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14876
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14877
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14878
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14879
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14880
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14881
2018-11-27 03:38:36 [DEBUG   ] 2018-11-27 03:38:36 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 14882
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14857 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14858 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14859 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14860 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14861 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14862 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14863 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14864 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14865 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14866 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14867 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14868 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14869 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14870 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14871 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14872 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14873 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14874 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14875 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14876 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14877 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14878 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14879 exited with status 0
2018-11-27 03:38:48 [DEBUG   ] 2018-11-27 03:38:48 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14880 exited with status 0
2018-11-27 03:38:49 [DEBUG   ] 2018-11-27 03:38:49 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14881 exited with status 0
2018-11-27 03:38:49 [DEBUG   ] 2018-11-27 03:38:49 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 14882 exited with status 0
2018-11-27 03:38:49 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] spawn_process: at 2018-11-27 03:38:49 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2018-11-27 03:38:49 [DEBUG   ] 2018-11-27 03:38:49 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 14909
2018-11-27 03:38:49 [DEBUG   ] 2018-11-27 03:38:49 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 14910
2018-11-27 03:38:54 [DEBUG   ] 2018-11-27 03:38:54 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 14909 exited with status 0
2018-11-27 03:38:54 [DEBUG   ] 2018-11-27 03:38:54 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 14910 exited with status 0
2018-11-27 03:38:54 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2018-11-27 03:38:54 [DEBUG   ] spawn_process: at 2018-11-27 03:38:54 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2018-11-27 03:38:54 [DEBUG   ] spawn_process: at 2018-11-27 03:38:54 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2018-11-27 03:38:54 [INFO    ] Setting Beam 1 xbars
2018-11-27 03:38:54 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2018-11-27 03:38:54 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2018-11-27 03:38:54 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2018-11-27 03:38:54 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2018-11-27 03:38:55 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2018-11-27 03:38:55 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2018-11-27 03:38:55 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2018-11-27 03:38:55 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2018-11-27 03:38:55 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2018-11-27 03:38:55 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2018-11-27 03:38:55 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2018-11-27 03:38:55 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2018-11-27 03:38:55 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2018-11-27 03:38:56 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2018-11-27 03:38:56 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2018-11-27 03:38:56 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2018-11-27 03:38:56 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2018-11-27 03:38:56 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2018-11-27 03:38:56 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2018-11-27 03:38:56 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2018-11-27 03:38:56 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2018-11-27 03:38:56 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2018-11-27 03:38:57 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2018-11-27 03:38:57 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2018-11-27 03:38:57 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2018-11-27 03:38:57 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2018-11-27 03:38:57 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2018-11-27 03:38:57 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2018-11-27 03:38:57 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2018-11-27 03:38:57 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2018-11-27 03:38:57 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2018-11-27 03:38:58 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2018-11-27 03:38:58 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2018-11-27 03:38:58 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2018-11-27 03:38:58 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2018-11-27 03:38:58 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2018-11-27 03:38:58 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2018-11-27 03:38:58 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2018-11-27 03:38:58 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2018-11-27 03:38:58 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2018-11-27 03:38:58 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2018-11-27 03:38:59 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2018-11-27 03:38:59 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2018-11-27 03:38:59 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2018-11-27 03:38:59 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2018-11-27 03:38:59 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2018-11-27 03:38:59 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2018-11-27 03:38:59 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2018-11-27 03:38:59 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2018-11-27 03:38:59 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2018-11-27 03:39:00 [INFO    ] Setting Beam 2 xbars
2018-11-27 03:39:00 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2018-11-27 03:39:00 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2018-11-27 03:39:00 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2018-11-27 03:39:00 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2018-11-27 03:39:00 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2018-11-27 03:39:00 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2018-11-27 03:39:00 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2018-11-27 03:39:00 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2018-11-27 03:39:00 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2018-11-27 03:39:01 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2018-11-27 03:39:01 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2018-11-27 03:39:01 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2018-11-27 03:39:01 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2018-11-27 03:39:01 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2018-11-27 03:39:01 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2018-11-27 03:39:01 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2018-11-27 03:39:01 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2018-11-27 03:39:01 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2018-11-27 03:39:01 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2018-11-27 03:39:02 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2018-11-27 03:39:02 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2018-11-27 03:39:02 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2018-11-27 03:39:02 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2018-11-27 03:39:02 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2018-11-27 03:39:02 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2018-11-27 03:39:02 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2018-11-27 03:39:02 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2018-11-27 03:39:02 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2018-11-27 03:39:03 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2018-11-27 03:39:03 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2018-11-27 03:39:03 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2018-11-27 03:39:03 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2018-11-27 03:39:03 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2018-11-27 03:39:03 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2018-11-27 03:39:03 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2018-11-27 03:39:03 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2018-11-27 03:39:03 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2018-11-27 03:39:04 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2018-11-27 03:39:04 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2018-11-27 03:39:04 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2018-11-27 03:39:04 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2018-11-27 03:39:04 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2018-11-27 03:39:04 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2018-11-27 03:39:04 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2018-11-27 03:39:04 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2018-11-27 03:39:04 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2018-11-27 03:39:05 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2018-11-27 03:39:05 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2018-11-27 03:39:05 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2018-11-27 03:39:05 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2018-11-27 03:39:05 [INFO    ] Setting Beam 3 xbars
2018-11-27 03:39:05 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2018-11-27 03:39:05 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2018-11-27 03:39:05 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2018-11-27 03:39:05 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2018-11-27 03:39:05 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2018-11-27 03:39:06 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2018-11-27 03:39:06 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2018-11-27 03:39:06 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2018-11-27 03:39:06 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2018-11-27 03:39:06 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2018-11-27 03:39:06 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2018-11-27 03:39:06 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2018-11-27 03:39:06 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2018-11-27 03:39:06 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2018-11-27 03:39:06 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2018-11-27 03:39:07 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2018-11-27 03:39:07 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2018-11-27 03:39:07 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2018-11-27 03:39:07 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2018-11-27 03:39:07 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2018-11-27 03:39:07 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2018-11-27 03:39:07 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2018-11-27 03:39:07 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2018-11-27 03:39:07 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2018-11-27 03:39:08 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2018-11-27 03:39:08 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2018-11-27 03:39:08 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2018-11-27 03:39:08 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2018-11-27 03:39:08 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2018-11-27 03:39:08 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2018-11-27 03:39:08 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2018-11-27 03:39:08 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2018-11-27 03:39:08 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2018-11-27 03:39:09 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2018-11-27 03:39:09 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2018-11-27 03:39:09 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2018-11-27 03:39:09 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2018-11-27 03:39:09 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2018-11-27 03:39:09 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2018-11-27 03:39:09 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2018-11-27 03:39:09 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2018-11-27 03:39:09 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2018-11-27 03:39:10 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2018-11-27 03:39:10 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2018-11-27 03:39:10 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2018-11-27 03:39:10 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2018-11-27 03:39:10 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2018-11-27 03:39:10 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2018-11-27 03:39:10 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2018-11-27 03:39:10 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2018-11-27 03:39:10 [INFO    ] Setting Beam 4 xbars
2018-11-27 03:39:10 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2018-11-27 03:39:11 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 03:39:11 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 03:39:11 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 03:39:11 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 03:39:11 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2018-11-27 03:39:11 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2018-11-27 03:39:11 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2018-11-27 03:39:11 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2018-11-27 03:39:11 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2018-11-27 03:39:12 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2018-11-27 03:39:12 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2018-11-27 03:39:12 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2018-11-27 03:39:12 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2018-11-27 03:39:12 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2018-11-27 03:39:12 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2018-11-27 03:39:12 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2018-11-27 03:39:12 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2018-11-27 03:39:12 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2018-11-27 03:39:12 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2018-11-27 03:39:13 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2018-11-27 03:39:13 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2018-11-27 03:39:13 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2018-11-27 03:39:13 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2018-11-27 03:39:13 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2018-11-27 03:39:13 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2018-11-27 03:39:13 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2018-11-27 03:39:13 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2018-11-27 03:39:13 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2018-11-27 03:39:14 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2018-11-27 03:39:14 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2018-11-27 03:39:14 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2018-11-27 03:39:14 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2018-11-27 03:39:14 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2018-11-27 03:39:14 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2018-11-27 03:39:14 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2018-11-27 03:39:14 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2018-11-27 03:39:14 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2018-11-27 03:39:15 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2018-11-27 03:39:15 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2018-11-27 03:39:15 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2018-11-27 03:39:15 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2018-11-27 03:39:15 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2018-11-27 03:39:15 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2018-11-27 03:39:15 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2018-11-27 03:39:15 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2018-11-27 03:39:15 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2018-11-27 03:39:16 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2018-11-27 03:39:16 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2018-11-27 03:39:16 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2018-11-27 03:39:16 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2018-11-27 03:39:16 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2018-11-27 03:39:16 [INFO    ] All xbars set
2018-11-27 03:39:16 [INFO    ] Setting adder chain routing for all beams
2018-11-27 03:39:21 [INFO    ] Setting beam ids
2018-11-27 03:39:31 [INFO    ] Starting beam delay calibration
2018-11-27 03:39:43 [INFO    ] Beam 1 latency is 5729
2018-11-27 03:39:43 [INFO    ] Beam 2 latency is 5764
2018-11-27 03:39:43 [INFO    ] Beam 3 latency is 5744
2018-11-27 03:39:43 [INFO    ] Beam 4 latency is 5736
2018-11-27 03:39:43 [INFO    ] Stop calibration
2018-11-27 03:39:51 [INFO    ] Beam delay calibration successful
2018-11-27 03:39:51 [INFO    ] Calibration passed
2018-11-27 03:39:51 [DEBUG   ] spawn_process: at 2018-11-27 03:39:51 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2018-11-27 03:39:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2018-11-27 03:39:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2018-11-27 03:39:51 [DEBUG   ] spawn_process: at 2018-11-27 03:39:51 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2018-11-27 03:39:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2018-11-27 03:39:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2018-11-27 03:39:51 [DEBUG   ] spawn_process: at 2018-11-27 03:39:51 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2018-11-27 03:39:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2018-11-27 03:39:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2018-11-27 03:39:51 [DEBUG   ] spawn_process: at 2018-11-27 03:39:51 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2018-11-27 03:39:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2018-11-27 03:39:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2018-11-27 03:39:51 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 03:39:52 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 279 ms (1543289992.281 s) with PID 16500 and time 1543289993
2018-11-27 03:39:52 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 03:39:52 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 366 ms (1543289992.369 s) with PID 16503 and time 1543289993
2018-11-27 03:39:53 [DEBUG   ] Finished updating ant stats in 0.879 seconds
2018-11-27 03:39:53 [INFO    ] Started antennas statistics background thread
2018-11-27 03:39:53 [INFO    ] Started board flags background thread
2018-11-27 03:39:53 [INFO    ] Started BAM background thread
2018-11-27 03:39:53 [INFO    ] Finished the INI process (ref. #639745) in 108.060 s
2018-11-27 03:39:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:39:54 [WARNING ] (13, 1, 'bot')
2018-11-27 03:39:54 [WARNING ] (13, 1, 'bot')
2018-11-27 03:39:54 [WARNING ] (13, 1, 'bot')
2018-11-27 03:39:54 [WARNING ] (13, 1, 'bot')
2018-11-27 03:39:54 [CRITICAL] MGT underflow detected on beam(s) 1, 2, 3, 4
2018-11-27 03:39:54 [DEBUG   ] Finished updating board flags in 1.076 seconds
2018-11-27 03:40:13 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 03:40:33 [DEBUG   ] Finished updating ant stats in 0.594 seconds
2018-11-27 03:40:48 [DEBUG   ] Got command RPT from MCS: ref #639750
2018-11-27 03:40:48 [DEBUG   ] summary = ERROR
2018-11-27 03:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639750  13 58449 13248897 A  ERRORERROR'
2018-11-27 03:40:48 [DEBUG   ] Got command RPT from MCS: ref #639751
2018-11-27 03:40:48 [DEBUG   ] info = T_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639751  98 58449 13248898 A  ERRORT_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:40:53 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:40:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 03:40:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:40:54 [WARNING ] (13, 1, 'bot')
2018-11-27 03:40:54 [WARNING ] (13, 1, 'bot')
2018-11-27 03:40:54 [WARNING ] (13, 1, 'bot')
2018-11-27 03:40:54 [WARNING ] (13, 1, 'bot')
2018-11-27 03:40:54 [CRITICAL] MGT underflow detected on beam(s) 1, 2, 3, 4
2018-11-27 03:40:54 [DEBUG   ] Finished updating board flags in 0.978 seconds
2018-11-27 03:41:05 [DEBUG   ] Got command RPT from MCS: ref #639767
2018-11-27 03:41:05 [DEBUG   ] summary = ERROR
2018-11-27 03:41:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639767  13 58449 13265421 A  ERRORERROR'
2018-11-27 03:41:05 [DEBUG   ] Got command RPT from MCS: ref #639768
2018-11-27 03:41:05 [DEBUG   ] info = T_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:41:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639768  98 58449 13265422 A  ERRORT_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:41:10 [DEBUG   ] Got command INI from MCS: ref #639769
2018-11-27 03:41:10 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2018-11-27 03:41:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   639769   8 58449 13270430 ABOOTING'
2018-11-27 03:41:13 [DEBUG   ] Finished updating ant stats in 0.427 seconds
2018-11-27 03:41:31 [INFO    ] -------------------
2018-11-27 03:41:31 [INFO    ]  LWA board status
2018-11-27 03:41:31 [INFO    ] -------------------
2018-11-27 03:41:31 [INFO    ] DP 1: 2 lwa27
2018-11-27 03:41:31 [INFO    ] DP 1: 3 lwa24
2018-11-27 03:41:31 [INFO    ] DP 1: 4 lwa23
2018-11-27 03:41:31 [INFO    ] DP 1: 5 lwa18
2018-11-27 03:41:31 [INFO    ] DP 1: 6 lwa10
2018-11-27 03:41:31 [INFO    ] DP 1: 7 lwa32
2018-11-27 03:41:31 [INFO    ] DP 1: 8 lwa11
2018-11-27 03:41:31 [INFO    ] DP 1: 9 lwa30
2018-11-27 03:41:31 [INFO    ] DP 1: 10 lwa12
2018-11-27 03:41:31 [INFO    ] DP 1: 11 lwa8
2018-11-27 03:41:31 [INFO    ] DP 1: 12 lwa13
2018-11-27 03:41:31 [INFO    ] DP 1: 13 lwa26
2018-11-27 03:41:31 [INFO    ] DP 1: 14 lwa14
2018-11-27 03:41:31 [INFO    ] DP 1: 16 lwa22
2018-11-27 03:41:31 [INFO    ] DP 1: 17 lwa20
2018-11-27 03:41:31 [INFO    ] DP 1: 18 lwa1
2018-11-27 03:41:31 [INFO    ] DP 1: 19 lwa5
2018-11-27 03:41:31 [INFO    ] DP 1: 20 lwa25
2018-11-27 03:41:31 [INFO    ] DP 1: 21 lwa35
2018-11-27 03:41:31 [INFO    ] DP 1: 22 lwa19
2018-11-27 03:41:31 [INFO    ] DP 1: 23 lwa21
2018-11-27 03:41:31 [INFO    ] DP 1: 24 lwa6
2018-11-27 03:41:31 [INFO    ] DP 1: 25 lwa7
2018-11-27 03:41:31 [INFO    ] DP 1: 26 lwa33
2018-11-27 03:41:31 [INFO    ] DP 1: 27 lwa15
2018-11-27 03:41:31 [INFO    ] DP 1: 28 lwa28
2018-11-27 03:41:31 [INFO    ] DP 2: 1 lwa16
2018-11-27 03:41:31 [INFO    ] DP 2: 15 lwa34
2018-11-27 03:41:31 [INFO    ] Total of 28 boards
2018-11-27 03:41:31 [INFO    ] Total of 260 stands
2018-11-27 03:41:31 [INFO    ] -------------------
2018-11-27 03:41:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17068
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17069
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17070
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17071
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17072
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17073
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17074
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17075
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17076
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17077
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17078
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17079
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17080
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17081
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17082
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17083
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17084
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17085
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17086
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17087
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17088
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17089
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17090
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17091
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17092
2018-11-27 03:41:38 [DEBUG   ] 2018-11-27 03:41:38 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 17093
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17068 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17069 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17070 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17071 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17072 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17073 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17074 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17075 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17076 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17077 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17078 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17079 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17080 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17081 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17082 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17083 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17084 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17085 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17086 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17087 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17088 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17089 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17090 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17091 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17092 exited with status 0
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 17093 exited with status 0
2018-11-27 03:41:50 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] spawn_process: at 2018-11-27 03:41:50 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 17120
2018-11-27 03:41:50 [DEBUG   ] 2018-11-27 03:41:50 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 17121
2018-11-27 03:41:56 [DEBUG   ] 2018-11-27 03:41:56 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 17120 exited with status 0
2018-11-27 03:41:56 [DEBUG   ] 2018-11-27 03:41:56 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 17121 exited with status 0
2018-11-27 03:41:56 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2018-11-27 03:41:56 [DEBUG   ] spawn_process: at 2018-11-27 03:41:56 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2018-11-27 03:41:56 [DEBUG   ] spawn_process: at 2018-11-27 03:41:56 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2018-11-27 03:41:56 [INFO    ] Setting Beam 1 xbars
2018-11-27 03:41:56 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2018-11-27 03:41:56 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2018-11-27 03:41:56 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2018-11-27 03:41:56 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2018-11-27 03:41:56 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2018-11-27 03:41:57 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2018-11-27 03:41:57 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2018-11-27 03:41:57 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2018-11-27 03:41:57 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2018-11-27 03:41:57 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2018-11-27 03:41:57 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2018-11-27 03:41:57 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2018-11-27 03:41:57 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2018-11-27 03:41:57 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2018-11-27 03:41:58 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2018-11-27 03:41:58 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2018-11-27 03:41:58 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2018-11-27 03:41:58 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2018-11-27 03:41:58 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2018-11-27 03:41:58 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2018-11-27 03:41:58 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2018-11-27 03:41:58 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2018-11-27 03:41:58 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2018-11-27 03:41:59 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2018-11-27 03:41:59 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2018-11-27 03:41:59 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2018-11-27 03:41:59 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2018-11-27 03:41:59 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2018-11-27 03:41:59 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2018-11-27 03:41:59 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2018-11-27 03:41:59 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2018-11-27 03:41:59 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2018-11-27 03:41:59 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2018-11-27 03:42:00 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2018-11-27 03:42:00 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2018-11-27 03:42:00 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2018-11-27 03:42:00 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2018-11-27 03:42:00 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2018-11-27 03:42:00 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2018-11-27 03:42:00 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2018-11-27 03:42:00 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2018-11-27 03:42:00 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2018-11-27 03:42:01 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2018-11-27 03:42:01 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2018-11-27 03:42:01 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2018-11-27 03:42:01 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2018-11-27 03:42:01 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2018-11-27 03:42:01 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2018-11-27 03:42:01 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2018-11-27 03:42:01 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2018-11-27 03:42:01 [INFO    ] Setting Beam 2 xbars
2018-11-27 03:42:01 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2018-11-27 03:42:02 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2018-11-27 03:42:02 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2018-11-27 03:42:02 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2018-11-27 03:42:02 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2018-11-27 03:42:02 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2018-11-27 03:42:02 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2018-11-27 03:42:02 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2018-11-27 03:42:02 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2018-11-27 03:42:02 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2018-11-27 03:42:03 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2018-11-27 03:42:03 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2018-11-27 03:42:03 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2018-11-27 03:42:03 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2018-11-27 03:42:03 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2018-11-27 03:42:03 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2018-11-27 03:42:03 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2018-11-27 03:42:03 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2018-11-27 03:42:03 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2018-11-27 03:42:04 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2018-11-27 03:42:04 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2018-11-27 03:42:04 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2018-11-27 03:42:04 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2018-11-27 03:42:04 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2018-11-27 03:42:04 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2018-11-27 03:42:04 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2018-11-27 03:42:04 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2018-11-27 03:42:04 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2018-11-27 03:42:04 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2018-11-27 03:42:05 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2018-11-27 03:42:05 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2018-11-27 03:42:05 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2018-11-27 03:42:05 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2018-11-27 03:42:05 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2018-11-27 03:42:05 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2018-11-27 03:42:05 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2018-11-27 03:42:05 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2018-11-27 03:42:05 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2018-11-27 03:42:06 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2018-11-27 03:42:06 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2018-11-27 03:42:06 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2018-11-27 03:42:06 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2018-11-27 03:42:06 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2018-11-27 03:42:06 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2018-11-27 03:42:06 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2018-11-27 03:42:06 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2018-11-27 03:42:06 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2018-11-27 03:42:07 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2018-11-27 03:42:07 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2018-11-27 03:42:07 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2018-11-27 03:42:07 [INFO    ] Setting Beam 3 xbars
2018-11-27 03:42:07 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2018-11-27 03:42:07 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2018-11-27 03:42:07 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2018-11-27 03:42:07 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2018-11-27 03:42:07 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2018-11-27 03:42:07 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2018-11-27 03:42:07 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2018-11-27 03:42:08 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2018-11-27 03:42:08 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2018-11-27 03:42:08 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2018-11-27 03:42:08 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2018-11-27 03:42:08 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2018-11-27 03:42:08 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2018-11-27 03:42:08 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2018-11-27 03:42:08 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2018-11-27 03:42:08 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2018-11-27 03:42:09 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2018-11-27 03:42:09 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2018-11-27 03:42:09 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2018-11-27 03:42:09 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2018-11-27 03:42:09 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2018-11-27 03:42:09 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2018-11-27 03:42:09 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2018-11-27 03:42:09 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2018-11-27 03:42:09 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2018-11-27 03:42:10 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2018-11-27 03:42:10 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2018-11-27 03:42:10 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2018-11-27 03:42:10 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2018-11-27 03:42:10 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2018-11-27 03:42:10 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2018-11-27 03:42:10 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2018-11-27 03:42:10 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2018-11-27 03:42:10 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2018-11-27 03:42:11 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2018-11-27 03:42:11 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2018-11-27 03:42:11 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2018-11-27 03:42:11 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2018-11-27 03:42:11 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2018-11-27 03:42:11 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2018-11-27 03:42:11 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2018-11-27 03:42:11 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2018-11-27 03:42:11 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2018-11-27 03:42:11 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2018-11-27 03:42:12 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2018-11-27 03:42:12 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2018-11-27 03:42:12 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2018-11-27 03:42:12 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2018-11-27 03:42:12 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2018-11-27 03:42:12 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2018-11-27 03:42:12 [INFO    ] Setting Beam 4 xbars
2018-11-27 03:42:12 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2018-11-27 03:42:12 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 03:42:12 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 03:42:13 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 03:42:13 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 03:42:13 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2018-11-27 03:42:13 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2018-11-27 03:42:13 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2018-11-27 03:42:13 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2018-11-27 03:42:13 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2018-11-27 03:42:13 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2018-11-27 03:42:13 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2018-11-27 03:42:14 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2018-11-27 03:42:14 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2018-11-27 03:42:14 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2018-11-27 03:42:14 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2018-11-27 03:42:14 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2018-11-27 03:42:14 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2018-11-27 03:42:14 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2018-11-27 03:42:14 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2018-11-27 03:42:14 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2018-11-27 03:42:15 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2018-11-27 03:42:15 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2018-11-27 03:42:15 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2018-11-27 03:42:15 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2018-11-27 03:42:15 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2018-11-27 03:42:15 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2018-11-27 03:42:15 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2018-11-27 03:42:15 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2018-11-27 03:42:15 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2018-11-27 03:42:16 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2018-11-27 03:42:16 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2018-11-27 03:42:16 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2018-11-27 03:42:16 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2018-11-27 03:42:16 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2018-11-27 03:42:16 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2018-11-27 03:42:16 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2018-11-27 03:42:16 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2018-11-27 03:42:16 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2018-11-27 03:42:17 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2018-11-27 03:42:17 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2018-11-27 03:42:17 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2018-11-27 03:42:17 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2018-11-27 03:42:17 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2018-11-27 03:42:17 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2018-11-27 03:42:17 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2018-11-27 03:42:17 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2018-11-27 03:42:17 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2018-11-27 03:42:17 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2018-11-27 03:42:18 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2018-11-27 03:42:18 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2018-11-27 03:42:18 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2018-11-27 03:42:18 [INFO    ] All xbars set
2018-11-27 03:42:18 [INFO    ] Setting adder chain routing for all beams
2018-11-27 03:42:23 [INFO    ] Setting beam ids
2018-11-27 03:42:33 [INFO    ] Starting beam delay calibration
2018-11-27 03:42:45 [INFO    ] Beam 1 latency is 5738
2018-11-27 03:42:45 [INFO    ] Beam 2 latency is 5758
2018-11-27 03:42:45 [INFO    ] Beam 3 latency is 5722
2018-11-27 03:42:45 [INFO    ] Beam 4 latency is 5738
2018-11-27 03:42:45 [INFO    ] Stop calibration
2018-11-27 03:42:53 [INFO    ] Beam delay calibration successful
2018-11-27 03:42:53 [INFO    ] Calibration passed
2018-11-27 03:42:53 [DEBUG   ] spawn_process: at 2018-11-27 03:42:53 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2018-11-27 03:42:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2018-11-27 03:42:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2018-11-27 03:42:53 [DEBUG   ] spawn_process: at 2018-11-27 03:42:53 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2018-11-27 03:42:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2018-11-27 03:42:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2018-11-27 03:42:53 [DEBUG   ] spawn_process: at 2018-11-27 03:42:53 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2018-11-27 03:42:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2018-11-27 03:42:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2018-11-27 03:42:53 [DEBUG   ] spawn_process: at 2018-11-27 03:42:53 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2018-11-27 03:42:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2018-11-27 03:42:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2018-11-27 03:42:53 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 03:42:53 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 509 ms (1543290173.511 s) with PID 18686 and time 1543290174
2018-11-27 03:42:53 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -3 msec
2018-11-27 03:42:54 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 279 ms (1543290174.281 s) with PID 18708 and time 1543290175
2018-11-27 03:42:55 [DEBUG   ] Finished updating ant stats in 0.966 seconds
2018-11-27 03:42:55 [INFO    ] Started antennas statistics background thread
2018-11-27 03:42:55 [INFO    ] Started board flags background thread
2018-11-27 03:42:55 [INFO    ] Started BAM background thread
2018-11-27 03:42:55 [INFO    ] Finished the INI process (ref. #639769) in 104.978 s
2018-11-27 03:42:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:42:56 [DEBUG   ] Got command RPT from MCS: ref #639772
2018-11-27 03:42:56 [DEBUG   ] summary = NORMAL
2018-11-27 03:42:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639772  14 58449 13376110 A NORMALNORMAL'
2018-11-27 03:42:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:42:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:42:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:42:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:42:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:42:56 [DEBUG   ] Finished updating board flags in 1.151 seconds
2018-11-27 03:43:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 03:43:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 03:43:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 03:43:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:43:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:43:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:43:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:43:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:43:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:43:56 [DEBUG   ] Finished updating board flags in 1.009 seconds
2018-11-27 03:44:10 [DEBUG   ] Got command RPT from MCS: ref #639782
2018-11-27 03:44:10 [DEBUG   ] summary = ERROR
2018-11-27 03:44:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639782  13 58449 13450433 A  ERRORERROR'
2018-11-27 03:44:10 [DEBUG   ] Got command RPT from MCS: ref #639783
2018-11-27 03:44:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:44:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639783  77 58449 13450434 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:44:15 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 03:44:36 [DEBUG   ] Finished updating ant stats in 1.603 seconds
2018-11-27 03:44:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:44:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:44:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:44:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:44:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:44:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:44:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:44:56 [DEBUG   ] Finished updating board flags in 0.986 seconds
2018-11-27 03:45:16 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2018-11-27 03:45:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:45:48 [DEBUG   ] Got command RPT from MCS: ref #639784
2018-11-27 03:45:48 [DEBUG   ] summary = ERROR
2018-11-27 03:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639784  13 58449 13548901 A  ERRORERROR'
2018-11-27 03:45:48 [DEBUG   ] Got command RPT from MCS: ref #639785
2018-11-27 03:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639785  77 58449 13548902 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:45:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:45:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:45:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:45:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:45:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:45:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:45:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:45:56 [DEBUG   ] Finished updating board flags in 1.272 seconds
2018-11-27 03:46:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:46:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:46:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:46:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:46:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:46:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:46:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:46:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:46:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:46:57 [DEBUG   ] Finished updating board flags in 1.786 seconds
2018-11-27 03:47:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:47:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:47:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:47:56 [DEBUG   ] Got command RPT from MCS: ref #639805
2018-11-27 03:47:56 [DEBUG   ] summary = ERROR
2018-11-27 03:47:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639805  13 58449 13676190 A  ERRORERROR'
2018-11-27 03:47:56 [DEBUG   ] Got command RPT from MCS: ref #639806
2018-11-27 03:47:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:47:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639806  77 58449 13676382 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:47:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:47:56 [DEBUG   ] Got command RPT from MCS: ref #639807
2018-11-27 03:47:56 [DEBUG   ] summary = ERROR
2018-11-27 03:47:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639807  13 58449 13676586 A  ERRORERROR'
2018-11-27 03:47:56 [DEBUG   ] Got command RPT from MCS: ref #639808
2018-11-27 03:47:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:47:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639808  77 58449 13676789 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:47:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:47:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:47:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:47:56 [WARNING ] (13, 1, 'bot')
2018-11-27 03:47:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:47:56 [DEBUG   ] Finished updating board flags in 0.985 seconds
2018-11-27 03:47:56 [DEBUG   ] Got command RPT from MCS: ref #639809
2018-11-27 03:47:56 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:47:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639809  12 58449 13676985 A  ERRORB•™š'
2018-11-27 03:48:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:48:36 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2018-11-27 03:48:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:48:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:48:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:48:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:48:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:48:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:48:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:48:57 [DEBUG   ] Finished updating board flags in 1.425 seconds
2018-11-27 03:49:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:49:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:49:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 03:49:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:49:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:49:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:49:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:49:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:49:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:49:57 [DEBUG   ] Finished updating board flags in 1.348 seconds
2018-11-27 03:50:15 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 03:50:36 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2018-11-27 03:50:48 [DEBUG   ] Got command RPT from MCS: ref #639814
2018-11-27 03:50:48 [DEBUG   ] summary = ERROR
2018-11-27 03:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639814  13 58449 13848897 A  ERRORERROR'
2018-11-27 03:50:48 [DEBUG   ] Got command RPT from MCS: ref #639815
2018-11-27 03:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639815  77 58449 13848898 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:50:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:50:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:50:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:50:57 [DEBUG   ] Finished updating board flags in 1.212 seconds
2018-11-27 03:51:16 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2018-11-27 03:51:35 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:51:56 [DEBUG   ] Finished updating ant stats in 1.403 seconds
2018-11-27 03:51:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:51:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:51:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:51:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:51:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:51:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:51:57 [DEBUG   ] Finished updating board flags in 1.179 seconds
2018-11-27 03:52:15 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:52:36 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2018-11-27 03:52:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:52:56 [DEBUG   ] Got command RPT from MCS: ref #639837
2018-11-27 03:52:56 [DEBUG   ] summary = ERROR
2018-11-27 03:52:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639837  13 58449 13976269 A  ERRORERROR'
2018-11-27 03:52:56 [DEBUG   ] Got command RPT from MCS: ref #639838
2018-11-27 03:52:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:52:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639838  77 58449 13976474 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:52:56 [DEBUG   ] Got command RPT from MCS: ref #639839
2018-11-27 03:52:56 [DEBUG   ] summary = ERROR
2018-11-27 03:52:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639839  13 58449 13976670 A  ERRORERROR'
2018-11-27 03:52:56 [DEBUG   ] Got command RPT from MCS: ref #639841
2018-11-27 03:52:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:52:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639841  77 58449 13976878 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:52:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:52:57 [DEBUG   ] Got command RPT from MCS: ref #639843
2018-11-27 03:52:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:52:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639843  12 58449 13977082 A  ERRORB�ÌÍ'
2018-11-27 03:52:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 2
2018-11-27 03:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:52:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:52:57 [DEBUG   ] Finished updating board flags in 1.053 seconds
2018-11-27 03:53:15 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:53:35 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:53:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:53:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:53:58 [WARNING ] (13, 1, 'bot')
2018-11-27 03:53:58 [WARNING ] (13, 1, 'bot')
2018-11-27 03:53:58 [WARNING ] (13, 1, 'bot')
2018-11-27 03:53:58 [WARNING ] (13, 1, 'bot')
2018-11-27 03:53:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:53:58 [DEBUG   ] Finished updating board flags in 2.004 seconds
2018-11-27 03:54:15 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:54:36 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2018-11-27 03:54:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:54:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:54:58 [WARNING ] (13, 1, 'bot')
2018-11-27 03:54:58 [WARNING ] (13, 1, 'bot')
2018-11-27 03:54:58 [WARNING ] (13, 1, 'bot')
2018-11-27 03:54:58 [WARNING ] (13, 1, 'bot')
2018-11-27 03:54:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:54:58 [DEBUG   ] Finished updating board flags in 2.048 seconds
2018-11-27 03:55:15 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:55:35 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:55:48 [DEBUG   ] Got command RPT from MCS: ref #639851
2018-11-27 03:55:48 [DEBUG   ] summary = ERROR
2018-11-27 03:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639851  13 58449 14148901 A  ERRORERROR'
2018-11-27 03:55:48 [DEBUG   ] Got command RPT from MCS: ref #639852
2018-11-27 03:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639852  77 58449 14148902 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:55:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:55:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:55:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:55:57 [DEBUG   ] Finished updating board flags in 1.349 seconds
2018-11-27 03:56:15 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:56:36 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2018-11-27 03:56:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:56:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 03:56:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:56:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:56:57 [DEBUG   ] Finished updating board flags in 1.238 seconds
2018-11-27 03:57:16 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2018-11-27 03:57:35 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:57:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 03:57:56 [DEBUG   ] Got command RPT from MCS: ref #639868
2018-11-27 03:57:56 [DEBUG   ] summary = ERROR
2018-11-27 03:57:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639868  13 58449 14276361 A  ERRORERROR'
2018-11-27 03:57:56 [DEBUG   ] Got command RPT from MCS: ref #639869
2018-11-27 03:57:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:57:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639869  77 58449 14276553 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:57:56 [DEBUG   ] Got command RPT from MCS: ref #639870
2018-11-27 03:57:56 [DEBUG   ] summary = ERROR
2018-11-27 03:57:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639870  13 58449 14276757 A  ERRORERROR'
2018-11-27 03:57:56 [DEBUG   ] Got command RPT from MCS: ref #639871
2018-11-27 03:57:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 03:57:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639871  77 58449 14276961 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 03:57:57 [DEBUG   ] Got command RPT from MCS: ref #639872
2018-11-27 03:57:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 03:57:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639872  12 58449 14277158 A  ERRORB„ÌÍ'
2018-11-27 03:57:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:57:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:57:57 [DEBUG   ] Finished updating board flags in 1.165 seconds
2018-11-27 03:58:15 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 03:58:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:58:45 [DEBUG   ] Got command DRX from MCS: ref #639882
2018-11-27 03:58:45 [INFO    ] sending drx command to lwa16
2018-11-27 03:58:45 [DEBUG   ] run_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 35100000.0 7 9 0']
2018-11-27 03:58:45 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 03:58:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   639882   8 58449 14325005 A  ERROR'
2018-11-27 03:58:45 [DEBUG   ] Got command DRX from MCS: ref #639881
2018-11-27 03:58:45 [INFO    ] sending drx command to lwa16
2018-11-27 03:58:45 [DEBUG   ] run_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 64500000.0 7 9 0']
2018-11-27 03:58:45 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 03:58:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   639881   8 58449 14325009 A  ERROR'
2018-11-27 03:58:45 [DEBUG   ] Got command DRX from MCS: ref #639883
2018-11-27 03:58:45 [INFO    ] sending drx command to lwa16
2018-11-27 03:58:45 [DEBUG   ] run_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 79200000.0 7 9 0']
2018-11-27 03:58:45 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 03:58:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   639883   8 58449 14325013 A  ERROR'
2018-11-27 03:58:45 [DEBUG   ] Got command DRX from MCS: ref #639884
2018-11-27 03:58:45 [INFO    ] sending drx command to lwa16
2018-11-27 03:58:45 [DEBUG   ] run_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 49800000.0 7 9 0']
2018-11-27 03:58:45 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 03:58:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   639884   8 58449 14325017 A  ERROR'
2018-11-27 03:58:45 [DEBUG   ] Got command BAM from MCS: ref #639885
2018-11-27 03:58:45 [INFO    ] Finished processing BAM request
2018-11-27 03:58:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 03:58:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639885   8 58449 14325033 A  ERROR'
2018-11-27 03:58:45 [DEBUG   ] Got command BAM from MCS: ref #639886
2018-11-27 03:58:45 [INFO    ] Finished processing BAM request
2018-11-27 03:58:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 03:58:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639886   8 58449 14325037 A  ERROR'
2018-11-27 03:58:45 [INFO    ] Finished starting DRX
2018-11-27 03:58:45 [INFO    ] Finished starting DRX
2018-11-27 03:58:45 [INFO    ] Finished starting DRX
2018-11-27 03:58:45 [INFO    ] Finished starting DRX
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 03:58:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 03:58:45 [DEBUG   ] spawn_process: at 2018-11-27 03:58:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 03:58:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:58:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:58:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 03:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:58:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:58:57 [DEBUG   ] Finished updating board flags in 1.013 seconds
2018-11-27 03:59:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:59:30 [DEBUG   ] Got command BAM from MCS: ref #639890
2018-11-27 03:59:30 [INFO    ] Finished processing BAM request
2018-11-27 03:59:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 03:59:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639890   8 58449 14370029 A  ERROR'
2018-11-27 03:59:30 [DEBUG   ] Got command BAM from MCS: ref #639889
2018-11-27 03:59:30 [INFO    ] Finished processing BAM request
2018-11-27 03:59:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 03:59:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639889   8 58449 14370033 A  ERROR'
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 03:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 03:59:30 [DEBUG   ] spawn_process: at 2018-11-27 03:59:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 03:59:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:59:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 03:59:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 03:59:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 03:59:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 03:59:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:59:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:59:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:59:57 [WARNING ] (13, 1, 'bot')
2018-11-27 03:59:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 03:59:57 [DEBUG   ] Finished updating board flags in 1.012 seconds
2018-11-27 04:00:15 [DEBUG   ] Got command BAM from MCS: ref #639892
2018-11-27 04:00:15 [INFO    ] Finished processing BAM request
2018-11-27 04:00:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:00:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639892   8 58449 14415029 A  ERROR'
2018-11-27 04:00:15 [DEBUG   ] Got command BAM from MCS: ref #639891
2018-11-27 04:00:15 [INFO    ] Finished processing BAM request
2018-11-27 04:00:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:00:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639891   8 58449 14415033 A  ERROR'
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 04:00:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 04:00:15 [DEBUG   ] spawn_process: at 2018-11-27 04:00:15 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 04:00:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:00:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:00:48 [DEBUG   ] Got command RPT from MCS: ref #639893
2018-11-27 04:00:48 [DEBUG   ] summary = ERROR
2018-11-27 04:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639893  13 58449 14448905 A  ERRORERROR'
2018-11-27 04:00:48 [DEBUG   ] Got command RPT from MCS: ref #639894
2018-11-27 04:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639894  77 58449 14448906 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:00:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:00:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:00:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:00:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:00:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:00:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:00:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:00:58 [DEBUG   ] Finished updating board flags in 1.615 seconds
2018-11-27 04:01:00 [DEBUG   ] Got command BAM from MCS: ref #639911
2018-11-27 04:01:00 [INFO    ] Finished processing BAM request
2018-11-27 04:01:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:01:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639911   8 58449 14460025 A  ERROR'
2018-11-27 04:01:00 [DEBUG   ] Got command BAM from MCS: ref #639910
2018-11-27 04:01:00 [INFO    ] Finished processing BAM request
2018-11-27 04:01:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:01:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639910   8 58449 14460029 A  ERROR'
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:01:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:01:00 [DEBUG   ] spawn_process: at 2018-11-27 04:01:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:01:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:01:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:01:45 [DEBUG   ] Got command BAM from MCS: ref #639917
2018-11-27 04:01:45 [INFO    ] Finished processing BAM request
2018-11-27 04:01:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:01:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639917   8 58449 14505033 A  ERROR'
2018-11-27 04:01:45 [DEBUG   ] Got command BAM from MCS: ref #639916
2018-11-27 04:01:45 [INFO    ] Finished processing BAM request
2018-11-27 04:01:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:01:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639916   8 58449 14505037 A  ERROR'
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:01:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:01:45 [DEBUG   ] spawn_process: at 2018-11-27 04:01:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:01:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:01:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:01:57 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:01:57 [WARNING ] (13, 1, 'bot')
2018-11-27 04:01:57 [WARNING ] (13, 1, 'bot')
2018-11-27 04:01:57 [WARNING ] (13, 1, 'bot')
2018-11-27 04:01:57 [WARNING ] (13, 1, 'bot')
2018-11-27 04:01:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:01:57 [DEBUG   ] Finished updating board flags in 0.969 seconds
2018-11-27 04:02:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:02:30 [DEBUG   ] Got command BAM from MCS: ref #639919
2018-11-27 04:02:30 [INFO    ] Finished processing BAM request
2018-11-27 04:02:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:02:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639919   8 58449 14550033 A  ERROR'
2018-11-27 04:02:30 [DEBUG   ] Got command BAM from MCS: ref #639918
2018-11-27 04:02:30 [INFO    ] Finished processing BAM request
2018-11-27 04:02:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:02:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639918   8 58449 14550037 A  ERROR'
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:02:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:02:30 [DEBUG   ] spawn_process: at 2018-11-27 04:02:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:02:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:02:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:02:56 [DEBUG   ] Got command RPT from MCS: ref #639922
2018-11-27 04:02:56 [DEBUG   ] summary = ERROR
2018-11-27 04:02:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639922  13 58449 14576449 A  ERRORERROR'
2018-11-27 04:02:56 [DEBUG   ] Got command RPT from MCS: ref #639923
2018-11-27 04:02:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:02:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639923  77 58449 14576641 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:02:56 [DEBUG   ] Got command RPT from MCS: ref #639924
2018-11-27 04:02:56 [DEBUG   ] summary = ERROR
2018-11-27 04:02:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639924  13 58449 14576845 A  ERRORERROR'
2018-11-27 04:02:57 [DEBUG   ] Got command RPT from MCS: ref #639926
2018-11-27 04:02:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:02:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639926  77 58449 14577049 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:02:57 [DEBUG   ] Got command RPT from MCS: ref #639928
2018-11-27 04:02:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:02:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639928  12 58449 14577253 A  ERRORBŠÌÍ'
2018-11-27 04:02:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:02:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-27 04:02:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:02:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 04:02:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:02:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:02:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:02:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:02:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:02:58 [DEBUG   ] Finished updating board flags in 1.373 seconds
2018-11-27 04:03:15 [DEBUG   ] Got command BAM from MCS: ref #639932
2018-11-27 04:03:15 [INFO    ] Finished processing BAM request
2018-11-27 04:03:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:03:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639932   8 58449 14595033 A  ERROR'
2018-11-27 04:03:15 [DEBUG   ] Got command BAM from MCS: ref #639933
2018-11-27 04:03:15 [INFO    ] Finished processing BAM request
2018-11-27 04:03:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:03:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639933   8 58449 14595037 A  ERROR'
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:03:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:03:15 [DEBUG   ] spawn_process: at 2018-11-27 04:03:15 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:03:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:03:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:03:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:03:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:03:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:03:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:03:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:03:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:03:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:03:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:03:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:03:58 [DEBUG   ] Finished updating board flags in 1.201 seconds
2018-11-27 04:04:00 [DEBUG   ] Got command BAM from MCS: ref #639935
2018-11-27 04:04:00 [INFO    ] Finished processing BAM request
2018-11-27 04:04:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:04:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639935   8 58449 14640033 A  ERROR'
2018-11-27 04:04:00 [DEBUG   ] Got command BAM from MCS: ref #639934
2018-11-27 04:04:00 [INFO    ] Finished processing BAM request
2018-11-27 04:04:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:04:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639934   8 58449 14640037 A  ERROR'
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:04:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:04:00 [DEBUG   ] spawn_process: at 2018-11-27 04:04:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:04:16 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 04:04:36 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 04:04:44 [DEBUG   ] Got command BAM from MCS: ref #639941
2018-11-27 04:04:44 [INFO    ] Finished processing BAM request
2018-11-27 04:04:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:04:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639941   8 58449 14684025 A  ERROR'
2018-11-27 04:04:44 [DEBUG   ] Got command BAM from MCS: ref #639940
2018-11-27 04:04:44 [INFO    ] Finished processing BAM request
2018-11-27 04:04:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:04:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639940   8 58449 14684029 A  ERROR'
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:04:44 [DEBUG   ] spawn_process: at 2018-11-27 04:04:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:04:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:04:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:04:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 04:04:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:04:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:04:58 [DEBUG   ] Finished updating board flags in 1.191 seconds
2018-11-27 04:05:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:05:28 [DEBUG   ] Got command BAM from MCS: ref #639942
2018-11-27 04:05:28 [INFO    ] Finished processing BAM request
2018-11-27 04:05:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:05:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639942   8 58449 14728033 A  ERROR'
2018-11-27 04:05:28 [DEBUG   ] Got command BAM from MCS: ref #639943
2018-11-27 04:05:28 [INFO    ] Finished processing BAM request
2018-11-27 04:05:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:05:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639943   8 58449 14728037 A  ERROR'
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:05:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:05:28 [DEBUG   ] spawn_process: at 2018-11-27 04:05:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:05:36 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 04:05:48 [DEBUG   ] Got command RPT from MCS: ref #639944
2018-11-27 04:05:48 [DEBUG   ] summary = ERROR
2018-11-27 04:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639944  13 58449 14748897 A  ERRORERROR'
2018-11-27 04:05:48 [DEBUG   ] Got command RPT from MCS: ref #639945
2018-11-27 04:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639945  77 58449 14748898 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:05:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:05:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:05:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 04:05:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:05:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:05:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:05:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:05:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:05:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:05:58 [DEBUG   ] Finished updating board flags in 0.998 seconds
2018-11-27 04:06:12 [DEBUG   ] Got command BAM from MCS: ref #639962
2018-11-27 04:06:12 [INFO    ] Finished processing BAM request
2018-11-27 04:06:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:06:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639962   8 58449 14772034 A  ERROR'
2018-11-27 04:06:12 [DEBUG   ] Got command BAM from MCS: ref #639961
2018-11-27 04:06:12 [INFO    ] Finished processing BAM request
2018-11-27 04:06:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:06:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639961   8 58449 14772038 A  ERROR'
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:06:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:06:12 [DEBUG   ] spawn_process: at 2018-11-27 04:06:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:06:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:06:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:06:56 [DEBUG   ] Got command BAM from MCS: ref #639963
2018-11-27 04:06:56 [INFO    ] Finished processing BAM request
2018-11-27 04:06:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:06:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639963   8 58449 14816025 A  ERROR'
2018-11-27 04:06:56 [DEBUG   ] Got command BAM from MCS: ref #639964
2018-11-27 04:06:56 [INFO    ] Finished processing BAM request
2018-11-27 04:06:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:06:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639964   8 58449 14816029 A  ERROR'
2018-11-27 04:06:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:06:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:06:56 [DEBUG   ] spawn_process: at 2018-11-27 04:06:56 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:06:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 04:06:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:06:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:06:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 04:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:06:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:06:58 [DEBUG   ] Finished updating board flags in 1.256 seconds
2018-11-27 04:07:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:07:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:07:40 [DEBUG   ] Got command BAM from MCS: ref #639969
2018-11-27 04:07:40 [INFO    ] Finished processing BAM request
2018-11-27 04:07:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:07:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639969   8 58449 14860025 A  ERROR'
2018-11-27 04:07:40 [DEBUG   ] Got command BAM from MCS: ref #639970
2018-11-27 04:07:40 [INFO    ] Finished processing BAM request
2018-11-27 04:07:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:07:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639970   8 58449 14860029 A  ERROR'
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:07:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:07:40 [DEBUG   ] spawn_process: at 2018-11-27 04:07:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:07:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:07:56 [DEBUG   ] Got command RPT from MCS: ref #639971
2018-11-27 04:07:56 [DEBUG   ] summary = ERROR
2018-11-27 04:07:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639971  13 58449 14876529 A  ERRORERROR'
2018-11-27 04:07:56 [DEBUG   ] Got command RPT from MCS: ref #639972
2018-11-27 04:07:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:07:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639972  77 58449 14876733 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:07:56 [DEBUG   ] Got command RPT from MCS: ref #639973
2018-11-27 04:07:56 [DEBUG   ] summary = ERROR
2018-11-27 04:07:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639973  13 58449 14876937 A  ERRORERROR'
2018-11-27 04:07:57 [DEBUG   ] Got command RPT from MCS: ref #639974
2018-11-27 04:07:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:07:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639974  77 58449 14877129 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:07:57 [DEBUG   ] Got command RPT from MCS: ref #639975
2018-11-27 04:07:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:07:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639975  12 58449 14877333 A  ERRORB ff'
2018-11-27 04:07:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:07:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2018-11-27 04:07:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:07:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:07:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:07:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:07:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:07:59 [DEBUG   ] Finished updating board flags in 1.836 seconds
2018-11-27 04:08:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:08:24 [DEBUG   ] Got command BAM from MCS: ref #639977
2018-11-27 04:08:24 [INFO    ] Finished processing BAM request
2018-11-27 04:08:24 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:08:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639977   8 58449 14904029 A  ERROR'
2018-11-27 04:08:24 [DEBUG   ] Got command BAM from MCS: ref #639976
2018-11-27 04:08:24 [INFO    ] Finished processing BAM request
2018-11-27 04:08:24 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:08:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639976   8 58449 14904033 A  ERROR'
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:08:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:08:24 [DEBUG   ] spawn_process: at 2018-11-27 04:08:24 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:08:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:08:56 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 04:08:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:08:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 04:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:08:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:08:58 [DEBUG   ] Finished updating board flags in 1.166 seconds
2018-11-27 04:09:08 [DEBUG   ] Got command BAM from MCS: ref #639979
2018-11-27 04:09:08 [INFO    ] Finished processing BAM request
2018-11-27 04:09:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:09:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639979   8 58449 14948029 A  ERROR'
2018-11-27 04:09:08 [DEBUG   ] Got command BAM from MCS: ref #639978
2018-11-27 04:09:08 [INFO    ] Finished processing BAM request
2018-11-27 04:09:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:09:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639978   8 58449 14948033 A  ERROR'
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:09:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:09:08 [DEBUG   ] spawn_process: at 2018-11-27 04:09:08 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:09:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:09:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:09:52 [DEBUG   ] Got command BAM from MCS: ref #639980
2018-11-27 04:09:52 [INFO    ] Finished processing BAM request
2018-11-27 04:09:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:09:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639980   8 58449 14992033 A  ERROR'
2018-11-27 04:09:52 [DEBUG   ] Got command BAM from MCS: ref #639981
2018-11-27 04:09:52 [INFO    ] Finished processing BAM request
2018-11-27 04:09:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:09:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639981   8 58449 14992037 A  ERROR'
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:09:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:09:52 [DEBUG   ] spawn_process: at 2018-11-27 04:09:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:09:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:09:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:09:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:09:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:09:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:09:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:09:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:09:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:09:58 [DEBUG   ] Finished updating board flags in 1.037 seconds
2018-11-27 04:10:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:10:36 [DEBUG   ] Got command BAM from MCS: ref #639987
2018-11-27 04:10:36 [INFO    ] Finished processing BAM request
2018-11-27 04:10:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:10:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639987   8 58449 15036025 A  ERROR'
2018-11-27 04:10:36 [DEBUG   ] Got command BAM from MCS: ref #639986
2018-11-27 04:10:36 [INFO    ] Finished processing BAM request
2018-11-27 04:10:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:10:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   639986   8 58449 15036029 A  ERROR'
2018-11-27 04:10:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:10:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:10:36 [DEBUG   ] spawn_process: at 2018-11-27 04:10:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:10:48 [DEBUG   ] Got command RPT from MCS: ref #639988
2018-11-27 04:10:48 [DEBUG   ] summary = ERROR
2018-11-27 04:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639988  13 58449 15048909 A  ERRORERROR'
2018-11-27 04:10:48 [DEBUG   ] Got command RPT from MCS: ref #639989
2018-11-27 04:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   639989  77 58449 15048910 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:10:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:10:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 04:10:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:10:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:10:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:10:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:10:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:10:58 [WARNING ] (13, 1, 'bot')
2018-11-27 04:10:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:10:58 [DEBUG   ] Finished updating board flags in 0.963 seconds
2018-11-27 04:11:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:11:20 [DEBUG   ] Got command BAM from MCS: ref #640006
2018-11-27 04:11:20 [INFO    ] Finished processing BAM request
2018-11-27 04:11:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:11:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640006   8 58449 15080029 A  ERROR'
2018-11-27 04:11:20 [DEBUG   ] Got command BAM from MCS: ref #640005
2018-11-27 04:11:20 [INFO    ] Finished processing BAM request
2018-11-27 04:11:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:11:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640005   8 58449 15080033 A  ERROR'
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:11:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:11:20 [DEBUG   ] spawn_process: at 2018-11-27 04:11:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:11:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:11:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:11:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:11:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:11:59 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:11:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:11:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:11:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:11:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:11:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:11:59 [DEBUG   ] Finished updating board flags in 1.367 seconds
2018-11-27 04:12:04 [DEBUG   ] Got command BAM from MCS: ref #640007
2018-11-27 04:12:04 [INFO    ] Finished processing BAM request
2018-11-27 04:12:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:12:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640007   8 58449 15124025 A  ERROR'
2018-11-27 04:12:04 [DEBUG   ] Got command BAM from MCS: ref #640008
2018-11-27 04:12:04 [INFO    ] Finished processing BAM request
2018-11-27 04:12:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:12:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640008   8 58449 15124029 A  ERROR'
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:12:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:12:04 [DEBUG   ] spawn_process: at 2018-11-27 04:12:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:12:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:12:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:12:48 [DEBUG   ] Got command BAM from MCS: ref #640011
2018-11-27 04:12:48 [INFO    ] Finished processing BAM request
2018-11-27 04:12:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640011   8 58449 15168033 A  ERROR'
2018-11-27 04:12:48 [DEBUG   ] Got command BAM from MCS: ref #640012
2018-11-27 04:12:48 [INFO    ] Finished processing BAM request
2018-11-27 04:12:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:12:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640012   8 58449 15168037 A  ERROR'
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 04:12:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 04:12:48 [DEBUG   ] spawn_process: at 2018-11-27 04:12:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 04:12:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:12:56 [DEBUG   ] Got command RPT from MCS: ref #640013
2018-11-27 04:12:56 [DEBUG   ] summary = ERROR
2018-11-27 04:12:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640013  13 58449 15176621 A  ERRORERROR'
2018-11-27 04:12:56 [DEBUG   ] Got command RPT from MCS: ref #640014
2018-11-27 04:12:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:12:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640014  77 58449 15176825 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:12:57 [DEBUG   ] Got command RPT from MCS: ref #640015
2018-11-27 04:12:57 [DEBUG   ] summary = ERROR
2018-11-27 04:12:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640015  13 58449 15177017 A  ERRORERROR'
2018-11-27 04:12:57 [DEBUG   ] Got command RPT from MCS: ref #640017
2018-11-27 04:12:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:12:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640017  77 58449 15177221 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:12:57 [DEBUG   ] Got command RPT from MCS: ref #640019
2018-11-27 04:12:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:12:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640019  12 58449 15177425 A  ERRORB ff'
2018-11-27 04:12:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:12:59 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:12:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:12:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:12:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:12:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:12:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:12:59 [DEBUG   ] Finished updating board flags in 1.188 seconds
2018-11-27 04:13:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:13:32 [DEBUG   ] Got command BAM from MCS: ref #640027
2018-11-27 04:13:32 [INFO    ] Finished processing BAM request
2018-11-27 04:13:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:13:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640027   8 58449 15212033 A  ERROR'
2018-11-27 04:13:32 [DEBUG   ] Got command BAM from MCS: ref #640028
2018-11-27 04:13:32 [INFO    ] Finished processing BAM request
2018-11-27 04:13:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:13:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640028   8 58449 15212037 A  ERROR'
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:13:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:13:32 [DEBUG   ] spawn_process: at 2018-11-27 04:13:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:13:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:13:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:13:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 04:13:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:13:59 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:13:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:13:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:13:59 [DEBUG   ] Finished updating board flags in 1.018 seconds
2018-11-27 04:14:16 [DEBUG   ] Got command BAM from MCS: ref #640029
2018-11-27 04:14:16 [INFO    ] Finished processing BAM request
2018-11-27 04:14:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:14:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640029   8 58449 15256033 A  ERROR'
2018-11-27 04:14:16 [DEBUG   ] Got command BAM from MCS: ref #640030
2018-11-27 04:14:16 [INFO    ] Finished processing BAM request
2018-11-27 04:14:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:14:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640030   8 58449 15256037 A  ERROR'
2018-11-27 04:14:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:14:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:14:16 [DEBUG   ] spawn_process: at 2018-11-27 04:14:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:14:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:14:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:14:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 04:14:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:14:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 04:14:59 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:14:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:14:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:14:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:14:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:14:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:14:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:14:59 [DEBUG   ] Finished updating board flags in 1.073 seconds
2018-11-27 04:15:00 [DEBUG   ] Got command BAM from MCS: ref #640032
2018-11-27 04:15:00 [INFO    ] Finished processing BAM request
2018-11-27 04:15:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:15:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640032   8 58449 15300025 A  ERROR'
2018-11-27 04:15:00 [DEBUG   ] Got command BAM from MCS: ref #640031
2018-11-27 04:15:00 [INFO    ] Finished processing BAM request
2018-11-27 04:15:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:15:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640031   8 58449 15300029 A  ERROR'
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:15:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:15:00 [DEBUG   ] spawn_process: at 2018-11-27 04:15:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:15:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:15:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:15:44 [DEBUG   ] Got command BAM from MCS: ref #640034
2018-11-27 04:15:44 [INFO    ] Finished processing BAM request
2018-11-27 04:15:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:15:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640034   8 58449 15344029 A  ERROR'
2018-11-27 04:15:44 [DEBUG   ] Got command BAM from MCS: ref #640033
2018-11-27 04:15:44 [INFO    ] Finished processing BAM request
2018-11-27 04:15:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:15:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640033   8 58449 15344033 A  ERROR'
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:15:44 [DEBUG   ] spawn_process: at 2018-11-27 04:15:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:15:48 [DEBUG   ] Got command RPT from MCS: ref #640035
2018-11-27 04:15:48 [DEBUG   ] summary = ERROR
2018-11-27 04:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640035  13 58449 15348905 A  ERRORERROR'
2018-11-27 04:15:48 [DEBUG   ] Got command RPT from MCS: ref #640036
2018-11-27 04:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640036  77 58449 15348906 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:15:56 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 04:15:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 04:15:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 04:15:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:16:00 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:16:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 04:16:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:16:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:16:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:16:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:16:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:16:00 [DEBUG   ] Finished updating board flags in 1.994 seconds
2018-11-27 04:16:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:16:27 [DEBUG   ] Got command BAM from MCS: ref #640056
2018-11-27 04:16:27 [INFO    ] Finished processing BAM request
2018-11-27 04:16:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:16:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640056   8 58449 15387029 A  ERROR'
2018-11-27 04:16:27 [DEBUG   ] Got command BAM from MCS: ref #640057
2018-11-27 04:16:27 [INFO    ] Finished processing BAM request
2018-11-27 04:16:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:16:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640057   8 58449 15387033 A  ERROR'
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 04:16:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 04:16:27 [DEBUG   ] spawn_process: at 2018-11-27 04:16:27 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 04:16:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:16:56 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 04:16:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 04:16:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 04:16:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:16:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:16:59 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:16:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:16:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:16:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:16:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:16:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:16:59 [DEBUG   ] Finished updating board flags in 1.180 seconds
2018-11-27 04:17:10 [DEBUG   ] Got command BAM from MCS: ref #640059
2018-11-27 04:17:10 [INFO    ] Finished processing BAM request
2018-11-27 04:17:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:17:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640059   8 58449 15430385 A  ERROR'
2018-11-27 04:17:10 [DEBUG   ] Got command BAM from MCS: ref #640058
2018-11-27 04:17:10 [INFO    ] Finished processing BAM request
2018-11-27 04:17:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:17:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640058   8 58449 15430389 A  ERROR'
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:17:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:17:10 [DEBUG   ] spawn_process: at 2018-11-27 04:17:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:17:16 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 04:17:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:17:53 [DEBUG   ] Got command BAM from MCS: ref #640060
2018-11-27 04:17:53 [INFO    ] Finished processing BAM request
2018-11-27 04:17:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:17:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640060   8 58449 15473025 A  ERROR'
2018-11-27 04:17:53 [DEBUG   ] Got command BAM from MCS: ref #640061
2018-11-27 04:17:53 [INFO    ] Finished processing BAM request
2018-11-27 04:17:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:17:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640061   8 58449 15473029 A  ERROR'
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:17:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:17:53 [DEBUG   ] spawn_process: at 2018-11-27 04:17:53 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:17:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:17:56 [DEBUG   ] Got command RPT from MCS: ref #640062
2018-11-27 04:17:56 [DEBUG   ] summary = ERROR
2018-11-27 04:17:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640062  13 58449 15476705 A  ERRORERROR'
2018-11-27 04:17:56 [DEBUG   ] Got command RPT from MCS: ref #640063
2018-11-27 04:17:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:17:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640063  77 58449 15476909 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:17:57 [DEBUG   ] Got command RPT from MCS: ref #640064
2018-11-27 04:17:57 [DEBUG   ] summary = ERROR
2018-11-27 04:17:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640064  13 58449 15477113 A  ERRORERROR'
2018-11-27 04:17:57 [DEBUG   ] Got command RPT from MCS: ref #640065
2018-11-27 04:17:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:17:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640065  77 58449 15477305 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:17:57 [DEBUG   ] Got command RPT from MCS: ref #640066
2018-11-27 04:17:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:17:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640066  12 58449 15477509 A  ERRORB ff'
2018-11-27 04:17:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 04:17:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:17:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 04:17:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:17:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 04:17:59 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:17:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:17:59 [DEBUG   ] Finished updating board flags in 1.000 seconds
2018-11-27 04:18:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:18:36 [DEBUG   ] Got command BAM from MCS: ref #640067
2018-11-27 04:18:36 [INFO    ] Finished processing BAM request
2018-11-27 04:18:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:18:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640067   8 58449 15516029 A  ERROR'
2018-11-27 04:18:36 [DEBUG   ] Got command BAM from MCS: ref #640068
2018-11-27 04:18:36 [INFO    ] Finished processing BAM request
2018-11-27 04:18:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:18:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640068   8 58449 15516033 A  ERROR'
2018-11-27 04:18:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:18:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:18:36 [DEBUG   ] spawn_process: at 2018-11-27 04:18:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:18:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:18:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:18:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:18:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:18:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2018-11-27 04:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:18:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:18:59 [DEBUG   ] Finished updating board flags in 1.018 seconds
2018-11-27 04:19:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:19:19 [DEBUG   ] Got command BAM from MCS: ref #640074
2018-11-27 04:19:19 [INFO    ] Finished processing BAM request
2018-11-27 04:19:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:19:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640074   8 58449 15559033 A  ERROR'
2018-11-27 04:19:19 [DEBUG   ] Got command BAM from MCS: ref #640073
2018-11-27 04:19:19 [INFO    ] Finished processing BAM request
2018-11-27 04:19:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:19:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640073   8 58449 15559037 A  ERROR'
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:19:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:19:19 [DEBUG   ] spawn_process: at 2018-11-27 04:19:19 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:19:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:19:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:19:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 04:19:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 04:19:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:19:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:19:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 04:19:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:19:59 [DEBUG   ] Finished updating board flags in 0.995 seconds
2018-11-27 04:20:02 [DEBUG   ] Got command BAM from MCS: ref #640075
2018-11-27 04:20:02 [INFO    ] Finished processing BAM request
2018-11-27 04:20:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:20:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640075   8 58449 15602033 A  ERROR'
2018-11-27 04:20:02 [DEBUG   ] Got command BAM from MCS: ref #640076
2018-11-27 04:20:02 [INFO    ] Finished processing BAM request
2018-11-27 04:20:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:20:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640076   8 58449 15602037 A  ERROR'
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:20:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:20:02 [DEBUG   ] spawn_process: at 2018-11-27 04:20:02 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:20:16 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 04:20:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:20:45 [DEBUG   ] Got command BAM from MCS: ref #640078
2018-11-27 04:20:45 [INFO    ] Finished processing BAM request
2018-11-27 04:20:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:20:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640078   8 58449 15645029 A  ERROR'
2018-11-27 04:20:45 [DEBUG   ] Got command BAM from MCS: ref #640077
2018-11-27 04:20:45 [INFO    ] Finished processing BAM request
2018-11-27 04:20:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:20:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640077   8 58449 15645033 A  ERROR'
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:20:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:20:45 [DEBUG   ] spawn_process: at 2018-11-27 04:20:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:20:48 [DEBUG   ] Got command RPT from MCS: ref #640079
2018-11-27 04:20:48 [DEBUG   ] summary = ERROR
2018-11-27 04:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640079  13 58449 15648909 A  ERRORERROR'
2018-11-27 04:20:48 [DEBUG   ] Got command RPT from MCS: ref #640080
2018-11-27 04:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640080  77 58449 15648910 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:20:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:20:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:21:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:21:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 04:21:00 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:21:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:21:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:21:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:21:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:21:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:21:00 [DEBUG   ] Finished updating board flags in 1.355 seconds
2018-11-27 04:21:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:21:28 [DEBUG   ] Got command BAM from MCS: ref #640096
2018-11-27 04:21:28 [INFO    ] Finished processing BAM request
2018-11-27 04:21:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:21:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640096   8 58449 15688033 A  ERROR'
2018-11-27 04:21:28 [DEBUG   ] Got command BAM from MCS: ref #640097
2018-11-27 04:21:28 [INFO    ] Finished processing BAM request
2018-11-27 04:21:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:21:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640097   8 58449 15688037 A  ERROR'
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:21:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:21:28 [DEBUG   ] spawn_process: at 2018-11-27 04:21:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:21:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:21:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:21:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:21:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:22:00 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:22:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:22:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:22:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:22:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:22:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:22:00 [DEBUG   ] Finished updating board flags in 1.203 seconds
2018-11-27 04:22:11 [DEBUG   ] Got command BAM from MCS: ref #640103
2018-11-27 04:22:11 [INFO    ] Finished processing BAM request
2018-11-27 04:22:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:22:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640103   8 58449 15731033 A  ERROR'
2018-11-27 04:22:11 [DEBUG   ] Got command BAM from MCS: ref #640102
2018-11-27 04:22:11 [INFO    ] Finished processing BAM request
2018-11-27 04:22:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:22:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640102   8 58449 15731037 A  ERROR'
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:22:11 [DEBUG   ] spawn_process: at 2018-11-27 04:22:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:22:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:22:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:22:54 [DEBUG   ] Got command BAM from MCS: ref #640107
2018-11-27 04:22:54 [INFO    ] Finished processing BAM request
2018-11-27 04:22:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:22:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640107   8 58449 15774029 A  ERROR'
2018-11-27 04:22:54 [DEBUG   ] Got command BAM from MCS: ref #640106
2018-11-27 04:22:54 [INFO    ] Finished processing BAM request
2018-11-27 04:22:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:22:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640106   8 58449 15774033 A  ERROR'
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 04:22:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 04:22:54 [DEBUG   ] spawn_process: at 2018-11-27 04:22:54 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 04:22:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:22:56 [DEBUG   ] Got command RPT from MCS: ref #640108
2018-11-27 04:22:56 [DEBUG   ] summary = ERROR
2018-11-27 04:22:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640108  13 58449 15776797 A  ERRORERROR'
2018-11-27 04:22:56 [DEBUG   ] Got command RPT from MCS: ref #640109
2018-11-27 04:22:56 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:22:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640109  77 58449 15776989 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:22:57 [DEBUG   ] Got command RPT from MCS: ref #640110
2018-11-27 04:22:57 [DEBUG   ] summary = ERROR
2018-11-27 04:22:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640110  13 58449 15777193 A  ERRORERROR'
2018-11-27 04:22:57 [DEBUG   ] Got command RPT from MCS: ref #640112
2018-11-27 04:22:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:22:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640112  77 58449 15777397 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:22:57 [DEBUG   ] Got command RPT from MCS: ref #640114
2018-11-27 04:22:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:22:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640114  12 58449 15777601 A  ERRORB¢ff'
2018-11-27 04:22:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 04:22:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:22:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:22:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 04:23:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:23:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 04:23:00 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:23:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:23:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:23:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:23:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:23:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:23:00 [DEBUG   ] Finished updating board flags in 1.244 seconds
2018-11-27 04:23:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:23:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:23:37 [DEBUG   ] Got command BAM from MCS: ref #640119
2018-11-27 04:23:37 [INFO    ] Finished processing BAM request
2018-11-27 04:23:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:23:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640119   8 58449 15817025 A  ERROR'
2018-11-27 04:23:37 [DEBUG   ] Got command BAM from MCS: ref #640118
2018-11-27 04:23:37 [INFO    ] Finished processing BAM request
2018-11-27 04:23:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:23:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640118   8 58449 15817029 A  ERROR'
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:23:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:23:37 [DEBUG   ] spawn_process: at 2018-11-27 04:23:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:23:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:23:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:24:00 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:24:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:24:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:24:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:24:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:24:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:24:00 [DEBUG   ] Finished updating board flags in 1.145 seconds
2018-11-27 04:24:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:24:20 [DEBUG   ] Got command BAM from MCS: ref #640120
2018-11-27 04:24:20 [INFO    ] Finished processing BAM request
2018-11-27 04:24:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:24:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640120   8 58449 15860025 A  ERROR'
2018-11-27 04:24:20 [DEBUG   ] Got command BAM from MCS: ref #640121
2018-11-27 04:24:20 [INFO    ] Finished processing BAM request
2018-11-27 04:24:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:24:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640121   8 58449 15860029 A  ERROR'
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:24:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:24:20 [DEBUG   ] spawn_process: at 2018-11-27 04:24:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:24:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:24:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:24:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:25:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:25:00 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:25:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:25:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:25:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:25:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:25:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:25:00 [DEBUG   ] Finished updating board flags in 1.018 seconds
2018-11-27 04:25:03 [DEBUG   ] Got command BAM from MCS: ref #640122
2018-11-27 04:25:03 [INFO    ] Finished processing BAM request
2018-11-27 04:25:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:25:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640122   8 58449 15903029 A  ERROR'
2018-11-27 04:25:03 [DEBUG   ] Got command BAM from MCS: ref #640123
2018-11-27 04:25:03 [INFO    ] Finished processing BAM request
2018-11-27 04:25:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:25:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640123   8 58449 15903033 A  ERROR'
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 04:25:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 04:25:03 [DEBUG   ] spawn_process: at 2018-11-27 04:25:03 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 04:25:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:25:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:25:46 [DEBUG   ] Got command BAM from MCS: ref #640128
2018-11-27 04:25:46 [INFO    ] Finished processing BAM request
2018-11-27 04:25:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:25:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640128   8 58449 15946025 A  ERROR'
2018-11-27 04:25:46 [DEBUG   ] Got command BAM from MCS: ref #640129
2018-11-27 04:25:46 [INFO    ] Finished processing BAM request
2018-11-27 04:25:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:25:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640129   8 58449 15946029 A  ERROR'
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:25:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:25:46 [DEBUG   ] spawn_process: at 2018-11-27 04:25:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:25:48 [DEBUG   ] Got command RPT from MCS: ref #640130
2018-11-27 04:25:48 [DEBUG   ] summary = ERROR
2018-11-27 04:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640130  13 58449 15948913 A  ERRORERROR'
2018-11-27 04:25:48 [DEBUG   ] Got command RPT from MCS: ref #640131
2018-11-27 04:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640131  77 58449 15948914 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:25:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:25:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:25:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:26:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 04:26:00 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:26:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 04:26:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:26:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:26:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:26:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:26:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:26:00 [DEBUG   ] Finished updating board flags in 1.030 seconds
2018-11-27 04:26:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:26:29 [DEBUG   ] Got command BAM from MCS: ref #640147
2018-11-27 04:26:29 [INFO    ] Finished processing BAM request
2018-11-27 04:26:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:26:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640147   8 58449 15989033 A  ERROR'
2018-11-27 04:26:29 [DEBUG   ] Got command BAM from MCS: ref #640148
2018-11-27 04:26:29 [INFO    ] Finished processing BAM request
2018-11-27 04:26:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:26:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640148   8 58449 15989037 A  ERROR'
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:26:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:26:29 [DEBUG   ] spawn_process: at 2018-11-27 04:26:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:26:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:26:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:27:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:27:01 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:27:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:27:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:27:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:27:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:27:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:27:01 [DEBUG   ] Finished updating board flags in 1.999 seconds
2018-11-27 04:27:12 [DEBUG   ] Got command BAM from MCS: ref #640149
2018-11-27 04:27:12 [INFO    ] Finished processing BAM request
2018-11-27 04:27:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:27:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640149   8 58449 16032029 A  ERROR'
2018-11-27 04:27:12 [DEBUG   ] Got command BAM from MCS: ref #640150
2018-11-27 04:27:12 [INFO    ] Finished processing BAM request
2018-11-27 04:27:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:27:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640150   8 58449 16032033 A  ERROR'
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:27:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:27:12 [DEBUG   ] spawn_process: at 2018-11-27 04:27:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:27:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:27:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:27:55 [DEBUG   ] Got command BAM from MCS: ref #640151
2018-11-27 04:27:55 [INFO    ] Finished processing BAM request
2018-11-27 04:27:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:27:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640151   8 58449 16075025 A  ERROR'
2018-11-27 04:27:55 [DEBUG   ] Got command BAM from MCS: ref #640152
2018-11-27 04:27:55 [INFO    ] Finished processing BAM request
2018-11-27 04:27:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:27:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640152   8 58449 16075029 A  ERROR'
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:27:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:27:55 [DEBUG   ] spawn_process: at 2018-11-27 04:27:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:27:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:27:56 [DEBUG   ] Got command RPT from MCS: ref #640153
2018-11-27 04:27:56 [DEBUG   ] summary = ERROR
2018-11-27 04:27:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640153  13 58449 16076877 A  ERRORERROR'
2018-11-27 04:27:57 [DEBUG   ] Got command RPT from MCS: ref #640154
2018-11-27 04:27:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:27:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640154  77 58449 16077081 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:27:57 [DEBUG   ] Got command RPT from MCS: ref #640155
2018-11-27 04:27:57 [DEBUG   ] summary = ERROR
2018-11-27 04:27:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640155  13 58449 16077285 A  ERRORERROR'
2018-11-27 04:27:57 [DEBUG   ] Got command RPT from MCS: ref #640156
2018-11-27 04:27:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:27:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640156  77 58449 16077477 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:27:57 [DEBUG   ] Got command RPT from MCS: ref #640157
2018-11-27 04:27:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:27:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640157  12 58449 16077681 A  ERRORB¡ff'
2018-11-27 04:28:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 04:28:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:28:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:28:00 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:28:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:28:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:28:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:28:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:28:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:28:00 [DEBUG   ] Finished updating board flags in 1.225 seconds
2018-11-27 04:28:16 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 04:28:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:28:38 [DEBUG   ] Got command BAM from MCS: ref #640163
2018-11-27 04:28:38 [INFO    ] Finished processing BAM request
2018-11-27 04:28:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:28:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640163   8 58449 16118033 A  ERROR'
2018-11-27 04:28:38 [DEBUG   ] Got command BAM from MCS: ref #640162
2018-11-27 04:28:38 [INFO    ] Finished processing BAM request
2018-11-27 04:28:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:28:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640162   8 58449 16118037 A  ERROR'
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:28:38 [DEBUG   ] spawn_process: at 2018-11-27 04:28:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:28:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:29:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 04:29:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:29:00 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:29:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:29:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:29:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:29:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:29:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:29:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:29:00 [DEBUG   ] Finished updating board flags in 1.191 seconds
2018-11-27 04:29:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:29:21 [DEBUG   ] Got command BAM from MCS: ref #640165
2018-11-27 04:29:21 [INFO    ] Finished processing BAM request
2018-11-27 04:29:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:29:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640165   8 58449 16161025 A  ERROR'
2018-11-27 04:29:21 [DEBUG   ] Got command BAM from MCS: ref #640164
2018-11-27 04:29:21 [INFO    ] Finished processing BAM request
2018-11-27 04:29:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:29:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640164   8 58449 16161029 A  ERROR'
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:29:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:29:21 [DEBUG   ] spawn_process: at 2018-11-27 04:29:21 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:29:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:29:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:30:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:30:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 04:30:00 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:30:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:30:00 [DEBUG   ] Finished updating board flags in 1.022 seconds
2018-11-27 04:30:04 [DEBUG   ] Got command BAM from MCS: ref #640167
2018-11-27 04:30:04 [INFO    ] Finished processing BAM request
2018-11-27 04:30:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:30:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640167   8 58449 16204025 A  ERROR'
2018-11-27 04:30:04 [DEBUG   ] Got command BAM from MCS: ref #640166
2018-11-27 04:30:04 [INFO    ] Finished processing BAM request
2018-11-27 04:30:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:30:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640166   8 58449 16204029 A  ERROR'
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:30:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:30:04 [DEBUG   ] spawn_process: at 2018-11-27 04:30:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:30:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:30:37 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 04:30:47 [DEBUG   ] Got command BAM from MCS: ref #640169
2018-11-27 04:30:47 [INFO    ] Finished processing BAM request
2018-11-27 04:30:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:30:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640169   8 58449 16247033 A  ERROR'
2018-11-27 04:30:47 [DEBUG   ] Got command BAM from MCS: ref #640168
2018-11-27 04:30:47 [INFO    ] Finished processing BAM request
2018-11-27 04:30:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:30:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640168   8 58449 16247037 A  ERROR'
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:30:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:30:47 [DEBUG   ] spawn_process: at 2018-11-27 04:30:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:30:48 [DEBUG   ] Got command RPT from MCS: ref #640170
2018-11-27 04:30:48 [DEBUG   ] summary = ERROR
2018-11-27 04:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640170  13 58449 16248909 A  ERRORERROR'
2018-11-27 04:30:48 [DEBUG   ] Got command RPT from MCS: ref #640171
2018-11-27 04:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640171  77 58449 16248910 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:30:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:31:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:31:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:31:00 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 04:31:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:31:00 [DEBUG   ] Finished updating board flags in 0.972 seconds
2018-11-27 04:31:17 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:31:30 [DEBUG   ] Got command BAM from MCS: ref #640191
2018-11-27 04:31:30 [INFO    ] Finished processing BAM request
2018-11-27 04:31:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:31:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640191   8 58449 16290025 A  ERROR'
2018-11-27 04:31:30 [DEBUG   ] Got command BAM from MCS: ref #640192
2018-11-27 04:31:30 [INFO    ] Finished processing BAM request
2018-11-27 04:31:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:31:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640192   8 58449 16290029 A  ERROR'
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:31:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:31:30 [DEBUG   ] spawn_process: at 2018-11-27 04:31:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:31:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:31:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:32:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:32:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:32:01 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:32:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:32:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:32:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:32:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:32:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:32:01 [DEBUG   ] Finished updating board flags in 1.399 seconds
2018-11-27 04:32:13 [DEBUG   ] Got command BAM from MCS: ref #640193
2018-11-27 04:32:13 [INFO    ] Finished processing BAM request
2018-11-27 04:32:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:32:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640193   8 58449 16333033 A  ERROR'
2018-11-27 04:32:13 [DEBUG   ] Got command BAM from MCS: ref #640194
2018-11-27 04:32:13 [INFO    ] Finished processing BAM request
2018-11-27 04:32:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:32:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640194   8 58449 16333037 A  ERROR'
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:32:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:32:13 [DEBUG   ] spawn_process: at 2018-11-27 04:32:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:32:17 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:32:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:32:56 [DEBUG   ] Got command BAM from MCS: ref #640197
2018-11-27 04:32:56 [INFO    ] Finished processing BAM request
2018-11-27 04:32:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:32:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640197   8 58449 16376033 A  ERROR'
2018-11-27 04:32:56 [DEBUG   ] Got command BAM from MCS: ref #640198
2018-11-27 04:32:56 [INFO    ] Finished processing BAM request
2018-11-27 04:32:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:32:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640198   8 58449 16376037 A  ERROR'
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:32:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:32:56 [DEBUG   ] spawn_process: at 2018-11-27 04:32:56 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:32:56 [DEBUG   ] Got command RPT from MCS: ref #640199
2018-11-27 04:32:56 [DEBUG   ] summary = ERROR
2018-11-27 04:32:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640199  13 58449 16376973 A  ERRORERROR'
2018-11-27 04:32:57 [DEBUG   ] Got command RPT from MCS: ref #640200
2018-11-27 04:32:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:32:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640200  77 58449 16377165 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:32:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:32:57 [DEBUG   ] Got command RPT from MCS: ref #640201
2018-11-27 04:32:57 [DEBUG   ] summary = ERROR
2018-11-27 04:32:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640201  13 58449 16377369 A  ERRORERROR'
2018-11-27 04:32:57 [DEBUG   ] Got command RPT from MCS: ref #640203
2018-11-27 04:32:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:32:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640203  77 58449 16377573 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:32:57 [DEBUG   ] Got command RPT from MCS: ref #640205
2018-11-27 04:32:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:32:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640205  12 58449 16377765 A  ERRORB£ff'
2018-11-27 04:33:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:33:01 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:33:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2018-11-27 04:33:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:33:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:33:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:33:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:33:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:33:01 [DEBUG   ] Finished updating board flags in 1.158 seconds
2018-11-27 04:33:17 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:33:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:33:39 [DEBUG   ] Got command BAM from MCS: ref #640209
2018-11-27 04:33:39 [INFO    ] Finished processing BAM request
2018-11-27 04:33:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:33:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640209   8 58449 16419033 A  ERROR'
2018-11-27 04:33:39 [DEBUG   ] Got command BAM from MCS: ref #640210
2018-11-27 04:33:39 [INFO    ] Finished processing BAM request
2018-11-27 04:33:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:33:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640210   8 58449 16419037 A  ERROR'
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:33:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:33:39 [DEBUG   ] spawn_process: at 2018-11-27 04:33:39 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:33:57 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 04:34:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:34:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:34:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-27 04:34:01 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:34:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:34:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:34:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:34:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:34:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:34:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:34:01 [DEBUG   ] Finished updating board flags in 1.044 seconds
2018-11-27 04:34:17 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:34:22 [DEBUG   ] Got command BAM from MCS: ref #640216
2018-11-27 04:34:22 [INFO    ] Finished processing BAM request
2018-11-27 04:34:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:34:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640216   8 58449 16462025 A  ERROR'
2018-11-27 04:34:22 [DEBUG   ] Got command BAM from MCS: ref #640215
2018-11-27 04:34:22 [INFO    ] Finished processing BAM request
2018-11-27 04:34:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:34:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640215   8 58449 16462029 A  ERROR'
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:34:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:34:22 [DEBUG   ] spawn_process: at 2018-11-27 04:34:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:34:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:34:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:35:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:35:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:35:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:35:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:35:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:35:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:35:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:35:02 [DEBUG   ] Finished updating board flags in 2.007 seconds
2018-11-27 04:35:05 [DEBUG   ] Got command BAM from MCS: ref #640218
2018-11-27 04:35:05 [INFO    ] Finished processing BAM request
2018-11-27 04:35:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:35:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640218   8 58449 16505029 A  ERROR'
2018-11-27 04:35:05 [DEBUG   ] Got command BAM from MCS: ref #640217
2018-11-27 04:35:05 [INFO    ] Finished processing BAM request
2018-11-27 04:35:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:35:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640217   8 58449 16505033 A  ERROR'
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:35:05 [DEBUG   ] spawn_process: at 2018-11-27 04:35:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:35:17 [DEBUG   ] Finished updating ant stats in 0.994 seconds
2018-11-27 04:35:37 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 04:35:48 [DEBUG   ] Got command BAM from MCS: ref #640219
2018-11-27 04:35:48 [INFO    ] Finished processing BAM request
2018-11-27 04:35:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640219   8 58449 16548025 A  ERROR'
2018-11-27 04:35:48 [DEBUG   ] Got command BAM from MCS: ref #640220
2018-11-27 04:35:48 [INFO    ] Finished processing BAM request
2018-11-27 04:35:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640220   8 58449 16548029 A  ERROR'
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:35:48 [DEBUG   ] Got command RPT from MCS: ref #640221
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:35:48 [DEBUG   ] summary = ERROR
2018-11-27 04:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640221  13 58449 16548910 A  ERRORERROR'
2018-11-27 04:35:48 [DEBUG   ] Got command RPT from MCS: ref #640222
2018-11-27 04:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640222  77 58449 16548911 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:35:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:35:48 [DEBUG   ] spawn_process: at 2018-11-27 04:35:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:35:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:36:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:36:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:36:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:36:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:36:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:36:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:36:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:36:02 [DEBUG   ] Finished updating board flags in 1.963 seconds
2018-11-27 04:36:17 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 04:36:31 [DEBUG   ] Got command BAM from MCS: ref #640239
2018-11-27 04:36:31 [INFO    ] Finished processing BAM request
2018-11-27 04:36:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:36:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640239   8 58449 16591025 A  ERROR'
2018-11-27 04:36:31 [DEBUG   ] Got command BAM from MCS: ref #640238
2018-11-27 04:36:31 [INFO    ] Finished processing BAM request
2018-11-27 04:36:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:36:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640238   8 58449 16591029 A  ERROR'
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:36:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:36:31 [DEBUG   ] spawn_process: at 2018-11-27 04:36:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:36:37 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 04:36:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:37:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:37:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 04:37:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:37:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:37:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 04:37:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:37:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:37:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:37:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:37:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:37:01 [DEBUG   ] Finished updating board flags in 1.270 seconds
2018-11-27 04:37:14 [DEBUG   ] Got command BAM from MCS: ref #640245
2018-11-27 04:37:14 [INFO    ] Finished processing BAM request
2018-11-27 04:37:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:37:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640245   8 58449 16634029 A  ERROR'
2018-11-27 04:37:14 [DEBUG   ] Got command BAM from MCS: ref #640244
2018-11-27 04:37:14 [INFO    ] Finished processing BAM request
2018-11-27 04:37:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:37:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640244   8 58449 16634033 A  ERROR'
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:37:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:37:14 [DEBUG   ] spawn_process: at 2018-11-27 04:37:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:37:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:37:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 04:37:57 [DEBUG   ] Got command BAM from MCS: ref #640247
2018-11-27 04:37:57 [INFO    ] Finished processing BAM request
2018-11-27 04:37:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:37:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640247   8 58449 16677033 A  ERROR'
2018-11-27 04:37:57 [DEBUG   ] Got command BAM from MCS: ref #640246
2018-11-27 04:37:57 [INFO    ] Finished processing BAM request
2018-11-27 04:37:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:37:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640246   8 58449 16677037 A  ERROR'
2018-11-27 04:37:57 [DEBUG   ] Got command RPT from MCS: ref #640248
2018-11-27 04:37:57 [DEBUG   ] summary = ERROR
2018-11-27 04:37:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640248  13 58449 16677049 A  ERRORERROR'
2018-11-27 04:37:57 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 04:37:57 [DEBUG   ] Got command RPT from MCS: ref #640249
2018-11-27 04:37:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:37:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640249  77 58449 16677257 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:37:57 [DEBUG   ] Got command RPT from MCS: ref #640250
2018-11-27 04:37:57 [DEBUG   ] summary = ERROR
2018-11-27 04:37:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640250  13 58449 16677453 A  ERRORERROR'
2018-11-27 04:37:57 [DEBUG   ] Got command RPT from MCS: ref #640251
2018-11-27 04:37:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:37:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640251  77 58449 16677657 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:37:57 [DEBUG   ] Got command RPT from MCS: ref #640252
2018-11-27 04:37:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:37:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640252  12 58449 16677861 A  ERRORB¢ff'
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:37:57 [DEBUG   ] spawn_process: at 2018-11-27 04:37:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:38:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 04:38:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:38:01 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:38:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:38:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:38:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:38:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:38:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:38:01 [DEBUG   ] Finished updating board flags in 1.208 seconds
2018-11-27 04:38:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:38:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:38:40 [DEBUG   ] Got command BAM from MCS: ref #640253
2018-11-27 04:38:40 [INFO    ] Finished processing BAM request
2018-11-27 04:38:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:38:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640253   8 58449 16720033 A  ERROR'
2018-11-27 04:38:40 [DEBUG   ] Got command BAM from MCS: ref #640254
2018-11-27 04:38:40 [INFO    ] Finished processing BAM request
2018-11-27 04:38:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:38:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640254   8 58449 16720037 A  ERROR'
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:38:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:38:40 [DEBUG   ] spawn_process: at 2018-11-27 04:38:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:38:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:39:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 04:39:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 04:39:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:39:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:39:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 04:39:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:39:01 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:39:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:39:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:39:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:39:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:39:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:39:01 [DEBUG   ] Finished updating board flags in 1.201 seconds
2018-11-27 04:39:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:39:23 [DEBUG   ] Got command BAM from MCS: ref #640256
2018-11-27 04:39:23 [INFO    ] Finished processing BAM request
2018-11-27 04:39:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:39:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640256   8 58449 16763034 A  ERROR'
2018-11-27 04:39:23 [DEBUG   ] Got command BAM from MCS: ref #640255
2018-11-27 04:39:23 [INFO    ] Finished processing BAM request
2018-11-27 04:39:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:39:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640255   8 58449 16763038 A  ERROR'
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:39:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:39:23 [DEBUG   ] spawn_process: at 2018-11-27 04:39:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:39:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:39:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:40:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 04:40:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:40:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:40:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:40:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 04:40:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:40:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:40:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:40:01 [WARNING ] (13, 1, 'bot')
2018-11-27 04:40:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:40:01 [DEBUG   ] Finished updating board flags in 1.220 seconds
2018-11-27 04:40:06 [DEBUG   ] Got command BAM from MCS: ref #640258
2018-11-27 04:40:06 [INFO    ] Finished processing BAM request
2018-11-27 04:40:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:40:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640258   8 58449 16806033 A  ERROR'
2018-11-27 04:40:06 [DEBUG   ] Got command BAM from MCS: ref #640257
2018-11-27 04:40:06 [INFO    ] Finished processing BAM request
2018-11-27 04:40:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:40:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640257   8 58449 16806037 A  ERROR'
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:40:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:40:06 [DEBUG   ] spawn_process: at 2018-11-27 04:40:06 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:40:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:40:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:40:48 [DEBUG   ] Got command RPT from MCS: ref #640265
2018-11-27 04:40:48 [DEBUG   ] summary = ERROR
2018-11-27 04:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640265  13 58449 16848917 A  ERRORERROR'
2018-11-27 04:40:48 [DEBUG   ] Got command RPT from MCS: ref #640266
2018-11-27 04:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640266  77 58449 16848918 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:40:49 [DEBUG   ] Got command BAM from MCS: ref #640263
2018-11-27 04:40:49 [INFO    ] Finished processing BAM request
2018-11-27 04:40:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:40:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640263   8 58449 16849029 A  ERROR'
2018-11-27 04:40:49 [DEBUG   ] Got command BAM from MCS: ref #640264
2018-11-27 04:40:49 [INFO    ] Finished processing BAM request
2018-11-27 04:40:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:40:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640264   8 58449 16849033 A  ERROR'
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:40:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:40:49 [DEBUG   ] spawn_process: at 2018-11-27 04:40:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:40:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:41:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:41:02 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:41:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:41:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:41:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:41:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:41:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:41:02 [DEBUG   ] Finished updating board flags in 1.765 seconds
2018-11-27 04:41:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:41:32 [DEBUG   ] Got command BAM from MCS: ref #640282
2018-11-27 04:41:32 [INFO    ] Finished processing BAM request
2018-11-27 04:41:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:41:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640282   8 58449 16892033 A  ERROR'
2018-11-27 04:41:32 [DEBUG   ] Got command BAM from MCS: ref #640283
2018-11-27 04:41:32 [INFO    ] Finished processing BAM request
2018-11-27 04:41:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:41:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640283   8 58449 16892037 A  ERROR'
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:41:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:41:32 [DEBUG   ] spawn_process: at 2018-11-27 04:41:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:41:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:41:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:42:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:42:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:42:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:42:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:42:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:42:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:42:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:42:02 [DEBUG   ] Finished updating board flags in 1.541 seconds
2018-11-27 04:42:15 [DEBUG   ] Got command BAM from MCS: ref #640284
2018-11-27 04:42:15 [INFO    ] Finished processing BAM request
2018-11-27 04:42:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:42:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640284   8 58449 16935029 A  ERROR'
2018-11-27 04:42:15 [DEBUG   ] Got command BAM from MCS: ref #640285
2018-11-27 04:42:15 [INFO    ] Finished processing BAM request
2018-11-27 04:42:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:42:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640285   8 58449 16935033 A  ERROR'
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:42:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:42:15 [DEBUG   ] spawn_process: at 2018-11-27 04:42:15 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:42:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:42:37 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 04:42:57 [DEBUG   ] Got command RPT from MCS: ref #640290
2018-11-27 04:42:57 [DEBUG   ] summary = ERROR
2018-11-27 04:42:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640290  13 58449 16977145 A  ERRORERROR'
2018-11-27 04:42:57 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 04:42:57 [DEBUG   ] Got command RPT from MCS: ref #640291
2018-11-27 04:42:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:42:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640291  77 58449 16977337 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:42:57 [DEBUG   ] Got command RPT from MCS: ref #640292
2018-11-27 04:42:57 [DEBUG   ] summary = ERROR
2018-11-27 04:42:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640292  13 58449 16977541 A  ERRORERROR'
2018-11-27 04:42:57 [DEBUG   ] Got command RPT from MCS: ref #640294
2018-11-27 04:42:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:42:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640294  77 58449 16977745 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:42:57 [DEBUG   ] Got command RPT from MCS: ref #640296
2018-11-27 04:42:57 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:42:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640296  12 58449 16977949 A  ERRORB¢ff'
2018-11-27 04:42:58 [DEBUG   ] Got command BAM from MCS: ref #640288
2018-11-27 04:42:58 [INFO    ] Finished processing BAM request
2018-11-27 04:42:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:42:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640288   8 58449 16978025 A  ERROR'
2018-11-27 04:42:58 [DEBUG   ] Got command BAM from MCS: ref #640289
2018-11-27 04:42:58 [INFO    ] Finished processing BAM request
2018-11-27 04:42:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:42:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640289   8 58449 16978029 A  ERROR'
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 04:42:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 04:42:58 [DEBUG   ] spawn_process: at 2018-11-27 04:42:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 04:43:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:43:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:43:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 04:43:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:43:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:43:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:43:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:43:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:43:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:43:02 [DEBUG   ] Finished updating board flags in 1.407 seconds
2018-11-27 04:43:17 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 04:43:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:43:42 [DEBUG   ] Got command BAM from MCS: ref #640305
2018-11-27 04:43:42 [INFO    ] Finished processing BAM request
2018-11-27 04:43:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:43:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640305   8 58449 17022029 A  ERROR'
2018-11-27 04:43:42 [DEBUG   ] Got command BAM from MCS: ref #640304
2018-11-27 04:43:42 [INFO    ] Finished processing BAM request
2018-11-27 04:43:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:43:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640304   8 58449 17022033 A  ERROR'
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:43:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:43:42 [DEBUG   ] spawn_process: at 2018-11-27 04:43:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:43:57 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 04:44:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:44:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:44:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:44:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 04:44:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:44:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:44:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:44:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:44:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:44:02 [DEBUG   ] Finished updating board flags in 1.428 seconds
2018-11-27 04:44:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:44:26 [DEBUG   ] Got command BAM from MCS: ref #640306
2018-11-27 04:44:26 [INFO    ] Finished processing BAM request
2018-11-27 04:44:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:44:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640306   8 58449 17066033 A  ERROR'
2018-11-27 04:44:26 [DEBUG   ] Got command BAM from MCS: ref #640307
2018-11-27 04:44:26 [INFO    ] Finished processing BAM request
2018-11-27 04:44:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:44:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640307   8 58449 17066037 A  ERROR'
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:44:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:44:26 [DEBUG   ] spawn_process: at 2018-11-27 04:44:26 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:44:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:44:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:45:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:45:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:45:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 04:45:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:45:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:45:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:45:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:45:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:45:02 [DEBUG   ] Finished updating board flags in 1.392 seconds
2018-11-27 04:45:10 [DEBUG   ] Got command BAM from MCS: ref #640308
2018-11-27 04:45:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:45:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640308   8 58449 17110025 A  ERROR'
2018-11-27 04:45:10 [DEBUG   ] Got command BAM from MCS: ref #640309
2018-11-27 04:45:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:45:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640309   8 58449 17110029 A  ERROR'
2018-11-27 04:45:10 [INFO    ] Finished processing BAM request
2018-11-27 04:45:10 [INFO    ] Finished processing BAM request
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:45:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:45:10 [DEBUG   ] spawn_process: at 2018-11-27 04:45:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:45:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:45:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:45:48 [DEBUG   ] Got command RPT from MCS: ref #640310
2018-11-27 04:45:48 [DEBUG   ] summary = ERROR
2018-11-27 04:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640310  13 58449 17148909 A  ERRORERROR'
2018-11-27 04:45:48 [DEBUG   ] Got command RPT from MCS: ref #640311
2018-11-27 04:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640311  77 58449 17148910 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:45:54 [DEBUG   ] Got command BAM from MCS: ref #640327
2018-11-27 04:45:54 [INFO    ] Finished processing BAM request
2018-11-27 04:45:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:45:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640327   8 58449 17154029 A  ERROR'
2018-11-27 04:45:54 [DEBUG   ] Got command BAM from MCS: ref #640328
2018-11-27 04:45:54 [INFO    ] Finished processing BAM request
2018-11-27 04:45:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:45:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640328   8 58449 17154033 A  ERROR'
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:45:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:45:54 [DEBUG   ] spawn_process: at 2018-11-27 04:45:54 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:45:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:46:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa10, chip 2
2018-11-27 04:46:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:46:02 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:46:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:46:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:46:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:46:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:46:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:46:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:46:02 [DEBUG   ] Finished updating board flags in 1.189 seconds
2018-11-27 04:46:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:46:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:46:38 [DEBUG   ] Got command BAM from MCS: ref #640334
2018-11-27 04:46:38 [INFO    ] Finished processing BAM request
2018-11-27 04:46:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:46:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640334   8 58449 17198033 A  ERROR'
2018-11-27 04:46:38 [DEBUG   ] Got command BAM from MCS: ref #640333
2018-11-27 04:46:38 [INFO    ] Finished processing BAM request
2018-11-27 04:46:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:46:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640333   8 58449 17198037 A  ERROR'
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:46:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:46:38 [DEBUG   ] spawn_process: at 2018-11-27 04:46:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:46:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:47:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:47:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 04:47:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:47:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:47:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 04:47:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:47:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:47:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:47:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:47:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:47:02 [DEBUG   ] Finished updating board flags in 1.007 seconds
2018-11-27 04:47:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:47:22 [DEBUG   ] Got command BAM from MCS: ref #640336
2018-11-27 04:47:22 [INFO    ] Finished processing BAM request
2018-11-27 04:47:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:47:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640336   8 58449 17242025 A  ERROR'
2018-11-27 04:47:22 [DEBUG   ] Got command BAM from MCS: ref #640335
2018-11-27 04:47:22 [INFO    ] Finished processing BAM request
2018-11-27 04:47:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:47:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640335   8 58449 17242029 A  ERROR'
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:47:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:47:22 [DEBUG   ] spawn_process: at 2018-11-27 04:47:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:47:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 04:47:57 [DEBUG   ] Got command RPT from MCS: ref #640337
2018-11-27 04:47:57 [DEBUG   ] summary = ERROR
2018-11-27 04:47:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640337  13 58449 17277229 A  ERRORERROR'
2018-11-27 04:47:57 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 04:47:57 [DEBUG   ] Got command RPT from MCS: ref #640338
2018-11-27 04:47:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:47:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640338  77 58449 17277433 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:47:57 [DEBUG   ] Got command RPT from MCS: ref #640339
2018-11-27 04:47:57 [DEBUG   ] summary = ERROR
2018-11-27 04:47:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640339  13 58449 17277625 A  ERRORERROR'
2018-11-27 04:47:57 [DEBUG   ] Got command RPT from MCS: ref #640340
2018-11-27 04:47:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:47:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640340  77 58449 17277829 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:47:58 [DEBUG   ] Got command RPT from MCS: ref #640341
2018-11-27 04:47:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640341  12 58449 17278033 A  ERRORB¢ff'
2018-11-27 04:48:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:48:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:48:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 1
2018-11-27 04:48:02 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:48:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:48:02 [DEBUG   ] Finished updating board flags in 1.017 seconds
2018-11-27 04:48:06 [DEBUG   ] Got command BAM from MCS: ref #640343
2018-11-27 04:48:06 [INFO    ] Finished processing BAM request
2018-11-27 04:48:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:48:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640343   8 58449 17286033 A  ERROR'
2018-11-27 04:48:06 [DEBUG   ] Got command BAM from MCS: ref #640342
2018-11-27 04:48:06 [INFO    ] Finished processing BAM request
2018-11-27 04:48:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:48:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640342   8 58449 17286037 A  ERROR'
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:48:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:48:06 [DEBUG   ] spawn_process: at 2018-11-27 04:48:06 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:48:17 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:48:37 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:48:50 [DEBUG   ] Got command BAM from MCS: ref #640345
2018-11-27 04:48:50 [INFO    ] Finished processing BAM request
2018-11-27 04:48:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:48:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640345   8 58449 17330029 A  ERROR'
2018-11-27 04:48:50 [DEBUG   ] Got command BAM from MCS: ref #640344
2018-11-27 04:48:50 [INFO    ] Finished processing BAM request
2018-11-27 04:48:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:48:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640344   8 58449 17330033 A  ERROR'
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:48:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:48:50 [DEBUG   ] spawn_process: at 2018-11-27 04:48:50 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:48:57 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:49:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 04:49:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:49:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 1
2018-11-27 04:49:03 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:49:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:49:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 04:49:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:49:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:49:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:49:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:49:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:49:03 [DEBUG   ] Finished updating board flags in 1.985 seconds
2018-11-27 04:49:17 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 04:49:34 [DEBUG   ] Got command BAM from MCS: ref #640350
2018-11-27 04:49:34 [INFO    ] Finished processing BAM request
2018-11-27 04:49:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:49:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640350   8 58449 17374025 A  ERROR'
2018-11-27 04:49:34 [DEBUG   ] Got command BAM from MCS: ref #640351
2018-11-27 04:49:34 [INFO    ] Finished processing BAM request
2018-11-27 04:49:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:49:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640351   8 58449 17374029 A  ERROR'
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:49:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:49:34 [DEBUG   ] spawn_process: at 2018-11-27 04:49:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:49:37 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 04:49:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 04:50:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:50:02 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:50:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:50:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:50:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:50:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:50:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:50:02 [DEBUG   ] Finished updating board flags in 1.225 seconds
2018-11-27 04:50:18 [DEBUG   ] Got command BAM from MCS: ref #640353
2018-11-27 04:50:18 [INFO    ] Finished processing BAM request
2018-11-27 04:50:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:50:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640353   8 58449 17418025 A  ERROR'
2018-11-27 04:50:18 [DEBUG   ] Got command BAM from MCS: ref #640352
2018-11-27 04:50:18 [INFO    ] Finished processing BAM request
2018-11-27 04:50:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:50:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640352   8 58449 17418029 A  ERROR'
2018-11-27 04:50:18 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:50:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:50:18 [DEBUG   ] spawn_process: at 2018-11-27 04:50:18 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:50:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:50:48 [DEBUG   ] Got command RPT from MCS: ref #640354
2018-11-27 04:50:48 [DEBUG   ] summary = ERROR
2018-11-27 04:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640354  13 58449 17448909 A  ERRORERROR'
2018-11-27 04:50:48 [DEBUG   ] Got command RPT from MCS: ref #640355
2018-11-27 04:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640355  77 58449 17448910 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:50:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:51:02 [DEBUG   ] Got command BAM from MCS: ref #640371
2018-11-27 04:51:02 [INFO    ] Finished processing BAM request
2018-11-27 04:51:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:51:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640371   8 58449 17462037 A  ERROR'
2018-11-27 04:51:02 [DEBUG   ] Got command BAM from MCS: ref #640372
2018-11-27 04:51:02 [INFO    ] Finished processing BAM request
2018-11-27 04:51:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:51:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640372   8 58449 17462045 A  ERROR'
2018-11-27 04:51:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 04:51:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:51:02 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:51:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 04:51:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 04:51:02 [DEBUG   ] spawn_process: at 2018-11-27 04:51:02 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 04:51:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:51:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:51:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:51:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:51:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:51:02 [DEBUG   ] Finished updating board flags in 1.293 seconds
2018-11-27 04:51:18 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:51:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:51:46 [DEBUG   ] Got command BAM from MCS: ref #640373
2018-11-27 04:51:46 [INFO    ] Finished processing BAM request
2018-11-27 04:51:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:51:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640373   8 58449 17506025 A  ERROR'
2018-11-27 04:51:46 [DEBUG   ] Got command BAM from MCS: ref #640374
2018-11-27 04:51:46 [INFO    ] Finished processing BAM request
2018-11-27 04:51:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:51:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640374   8 58449 17506029 A  ERROR'
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:51:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:51:46 [DEBUG   ] spawn_process: at 2018-11-27 04:51:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:51:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 04:52:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:52:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 04:52:02 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:52:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:52:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:52:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:52:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:52:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:52:02 [DEBUG   ] Finished updating board flags in 1.202 seconds
2018-11-27 04:52:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 04:52:30 [DEBUG   ] Got command BAM from MCS: ref #640379
2018-11-27 04:52:30 [INFO    ] Finished processing BAM request
2018-11-27 04:52:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:52:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640379   8 58449 17550029 A  ERROR'
2018-11-27 04:52:30 [DEBUG   ] Got command BAM from MCS: ref #640380
2018-11-27 04:52:30 [INFO    ] Finished processing BAM request
2018-11-27 04:52:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:52:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640380   8 58449 17550033 A  ERROR'
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:52:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:52:30 [DEBUG   ] spawn_process: at 2018-11-27 04:52:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:52:38 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 04:52:57 [DEBUG   ] Got command RPT from MCS: ref #640383
2018-11-27 04:52:57 [DEBUG   ] summary = ERROR
2018-11-27 04:52:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640383  13 58449 17577317 A  ERRORERROR'
2018-11-27 04:52:57 [DEBUG   ] Got command RPT from MCS: ref #640384
2018-11-27 04:52:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:52:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640384  77 58449 17577521 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:52:57 [DEBUG   ] Got command RPT from MCS: ref #640385
2018-11-27 04:52:57 [DEBUG   ] summary = ERROR
2018-11-27 04:52:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640385  13 58449 17577713 A  ERRORERROR'
2018-11-27 04:52:57 [DEBUG   ] Got command RPT from MCS: ref #640387
2018-11-27 04:52:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:52:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640387  77 58449 17577917 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:52:58 [DEBUG   ] Got command RPT from MCS: ref #640388
2018-11-27 04:52:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:52:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640388  12 58449 17578121 A  ERRORB£ff'
2018-11-27 04:52:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:53:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:53:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:53:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:53:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 04:53:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:53:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:53:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:53:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:53:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:53:02 [DEBUG   ] Finished updating board flags in 1.137 seconds
2018-11-27 04:53:14 [DEBUG   ] Got command BAM from MCS: ref #640394
2018-11-27 04:53:14 [INFO    ] Finished processing BAM request
2018-11-27 04:53:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:53:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640394   8 58449 17594029 A  ERROR'
2018-11-27 04:53:14 [DEBUG   ] Got command BAM from MCS: ref #640393
2018-11-27 04:53:14 [INFO    ] Finished processing BAM request
2018-11-27 04:53:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:53:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640393   8 58449 17594033 A  ERROR'
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:53:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:53:14 [DEBUG   ] spawn_process: at 2018-11-27 04:53:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:53:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:53:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:53:58 [DEBUG   ] Got command BAM from MCS: ref #640396
2018-11-27 04:53:58 [INFO    ] Finished processing BAM request
2018-11-27 04:53:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640396   8 58449 17638025 A  ERROR'
2018-11-27 04:53:58 [DEBUG   ] Got command BAM from MCS: ref #640395
2018-11-27 04:53:58 [INFO    ] Finished processing BAM request
2018-11-27 04:53:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:53:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640395   8 58449 17638029 A  ERROR'
2018-11-27 04:53:58 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:53:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:53:58 [DEBUG   ] spawn_process: at 2018-11-27 04:53:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:54:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 04:54:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:54:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:54:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:54:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:54:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:54:02 [WARNING ] (13, 1, 'bot')
2018-11-27 04:54:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:54:02 [DEBUG   ] Finished updating board flags in 1.006 seconds
2018-11-27 04:54:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:54:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:54:42 [DEBUG   ] Got command BAM from MCS: ref #640398
2018-11-27 04:54:42 [INFO    ] Finished processing BAM request
2018-11-27 04:54:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:54:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640398   8 58449 17682025 A  ERROR'
2018-11-27 04:54:42 [DEBUG   ] Got command BAM from MCS: ref #640397
2018-11-27 04:54:42 [INFO    ] Finished processing BAM request
2018-11-27 04:54:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:54:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640397   8 58449 17682029 A  ERROR'
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:54:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:54:42 [DEBUG   ] spawn_process: at 2018-11-27 04:54:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:54:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:55:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:55:03 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:55:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:55:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:55:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:55:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:55:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:55:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:55:03 [DEBUG   ] Finished updating board flags in 1.544 seconds
2018-11-27 04:55:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:55:27 [DEBUG   ] Got command BAM from MCS: ref #640403
2018-11-27 04:55:27 [INFO    ] Finished processing BAM request
2018-11-27 04:55:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:55:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640403   8 58449 17727033 A  ERROR'
2018-11-27 04:55:27 [DEBUG   ] Got command BAM from MCS: ref #640404
2018-11-27 04:55:27 [INFO    ] Finished processing BAM request
2018-11-27 04:55:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:55:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640404   8 58449 17727037 A  ERROR'
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:55:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:55:27 [DEBUG   ] spawn_process: at 2018-11-27 04:55:27 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:55:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:55:48 [DEBUG   ] Got command RPT from MCS: ref #640405
2018-11-27 04:55:48 [DEBUG   ] summary = ERROR
2018-11-27 04:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640405  13 58449 17748921 A  ERRORERROR'
2018-11-27 04:55:48 [DEBUG   ] Got command RPT from MCS: ref #640406
2018-11-27 04:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640406  77 58449 17748922 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:55:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:56:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 04:56:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:56:03 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:56:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:56:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:56:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:56:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:56:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:56:03 [DEBUG   ] Finished updating board flags in 1.390 seconds
2018-11-27 04:56:12 [DEBUG   ] Got command BAM from MCS: ref #640422
2018-11-27 04:56:12 [INFO    ] Finished processing BAM request
2018-11-27 04:56:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:56:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640422   8 58449 17772033 A  ERROR'
2018-11-27 04:56:12 [DEBUG   ] Got command BAM from MCS: ref #640423
2018-11-27 04:56:12 [INFO    ] Finished processing BAM request
2018-11-27 04:56:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:56:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640423   8 58449 17772037 A  ERROR'
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 04:56:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 04:56:12 [DEBUG   ] spawn_process: at 2018-11-27 04:56:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 04:56:18 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 04:56:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:56:57 [DEBUG   ] Got command BAM from MCS: ref #640425
2018-11-27 04:56:57 [INFO    ] Finished processing BAM request
2018-11-27 04:56:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:56:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640425   8 58449 17817025 A  ERROR'
2018-11-27 04:56:57 [DEBUG   ] Got command BAM from MCS: ref #640424
2018-11-27 04:56:57 [INFO    ] Finished processing BAM request
2018-11-27 04:56:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:56:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640424   8 58449 17817029 A  ERROR'
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 04:56:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 04:56:57 [DEBUG   ] spawn_process: at 2018-11-27 04:56:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 04:56:58 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 04:57:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 5
2018-11-27 04:57:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:57:03 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:57:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:57:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:57:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:57:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:57:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:57:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:57:03 [DEBUG   ] Finished updating board flags in 1.210 seconds
2018-11-27 04:57:18 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 04:57:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:57:42 [DEBUG   ] Got command BAM from MCS: ref #640426
2018-11-27 04:57:42 [INFO    ] Finished processing BAM request
2018-11-27 04:57:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:57:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640426   8 58449 17862025 A  ERROR'
2018-11-27 04:57:42 [DEBUG   ] Got command BAM from MCS: ref #640427
2018-11-27 04:57:42 [INFO    ] Finished processing BAM request
2018-11-27 04:57:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:57:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640427   8 58449 17862029 A  ERROR'
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:57:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:57:42 [DEBUG   ] spawn_process: at 2018-11-27 04:57:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:57:57 [DEBUG   ] Got command RPT from MCS: ref #640428
2018-11-27 04:57:57 [DEBUG   ] summary = ERROR
2018-11-27 04:57:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640428  13 58449 17877401 A  ERRORERROR'
2018-11-27 04:57:57 [DEBUG   ] Got command RPT from MCS: ref #640429
2018-11-27 04:57:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:57:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640429  77 58449 17877605 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:57:57 [DEBUG   ] Got command RPT from MCS: ref #640430
2018-11-27 04:57:57 [DEBUG   ] summary = ERROR
2018-11-27 04:57:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640430  13 58449 17877809 A  ERRORERROR'
2018-11-27 04:57:58 [DEBUG   ] Got command RPT from MCS: ref #640431
2018-11-27 04:57:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 04:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640431  77 58449 17878001 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 04:57:58 [DEBUG   ] Got command RPT from MCS: ref #640432
2018-11-27 04:57:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 04:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640432  12 58449 17878205 A  ERRORB¢ff'
2018-11-27 04:57:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:58:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:58:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:58:03 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 04:58:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:58:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:58:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:58:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:58:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:58:03 [DEBUG   ] Finished updating board flags in 1.205 seconds
2018-11-27 04:58:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:58:27 [DEBUG   ] Got command BAM from MCS: ref #640438
2018-11-27 04:58:27 [INFO    ] Finished processing BAM request
2018-11-27 04:58:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:58:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640438   8 58449 17907029 A  ERROR'
2018-11-27 04:58:27 [DEBUG   ] Got command BAM from MCS: ref #640437
2018-11-27 04:58:27 [INFO    ] Finished processing BAM request
2018-11-27 04:58:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 04:58:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640437   8 58449 17907033 A  ERROR'
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 04:58:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 04:58:27 [DEBUG   ] spawn_process: at 2018-11-27 04:58:27 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 04:58:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:58:47 [DEBUG   ] Got command STP from MCS: ref #640440
2018-11-27 04:58:47 [INFO    ] Finished processing BAM request
2018-11-27 04:58:47 [INFO    ] Shutdown the adder chain on beam #4
2018-11-27 04:58:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   640440   8 58449 17927029 A  ERROR'
2018-11-27 04:58:47 [DEBUG   ] Got command STP from MCS: ref #640439
2018-11-27 04:58:47 [INFO    ] Finished processing BAM request
2018-11-27 04:58:47 [INFO    ] Shutdown the adder chain on beam #3
2018-11-27 04:58:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   640439   8 58449 17927033 A  ERROR'
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 04:58:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 04:58:47 [DEBUG   ] spawn_process: at 2018-11-27 04:58:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 04:58:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:59:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 04:59:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 04:59:03 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 04:59:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 04:59:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:59:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:59:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:59:03 [WARNING ] (13, 1, 'bot')
2018-11-27 04:59:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 04:59:03 [DEBUG   ] Finished updating board flags in 1.208 seconds
2018-11-27 04:59:18 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 04:59:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 04:59:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 05:00:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:00:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 05:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:00:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:00:03 [DEBUG   ] Finished updating board flags in 1.203 seconds
2018-11-27 05:00:18 [DEBUG   ] Finished updating ant stats in 0.986 seconds
2018-11-27 05:00:38 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 05:00:48 [DEBUG   ] Got command RPT from MCS: ref #640441
2018-11-27 05:00:48 [DEBUG   ] summary = ERROR
2018-11-27 05:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640441  13 58449 18048921 A  ERRORERROR'
2018-11-27 05:00:48 [DEBUG   ] Got command RPT from MCS: ref #640442
2018-11-27 05:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640442  77 58449 18048922 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:00:58 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:01:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:01:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 05:01:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:01:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:01:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:01:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:01:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:01:03 [DEBUG   ] Finished updating board flags in 1.189 seconds
2018-11-27 05:01:18 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:01:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:01:58 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:02:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 05:02:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:02:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:02:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:02:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:02:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:02:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:02:03 [DEBUG   ] Finished updating board flags in 1.073 seconds
2018-11-27 05:02:18 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:02:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:02:57 [DEBUG   ] Got command RPT from MCS: ref #640464
2018-11-27 05:02:57 [DEBUG   ] summary = ERROR
2018-11-27 05:02:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640464  13 58449 18177493 A  ERRORERROR'
2018-11-27 05:02:57 [DEBUG   ] Got command RPT from MCS: ref #640465
2018-11-27 05:02:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:02:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640465  77 58449 18177697 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:02:57 [DEBUG   ] Got command RPT from MCS: ref #640466
2018-11-27 05:02:57 [DEBUG   ] summary = ERROR
2018-11-27 05:02:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640466  13 58449 18177889 A  ERRORERROR'
2018-11-27 05:02:58 [DEBUG   ] Got command RPT from MCS: ref #640468
2018-11-27 05:02:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:02:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640468  77 58449 18178093 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:02:58 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:02:58 [DEBUG   ] Got command RPT from MCS: ref #640469
2018-11-27 05:02:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:02:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640469  12 58449 18178297 A  ERRORB›™š'
2018-11-27 05:03:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:03:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:03:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:03:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:03:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:03:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:03:03 [DEBUG   ] Finished updating board flags in 1.479 seconds
2018-11-27 05:03:18 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:03:38 [DEBUG   ] Finished updating ant stats in 0.582 seconds
2018-11-27 05:03:58 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:04:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:04:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:04:03 [DEBUG   ] Finished updating board flags in 1.291 seconds
2018-11-27 05:04:18 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:04:38 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:04:58 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:05:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:05:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:05:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:05:03 [DEBUG   ] Finished updating board flags in 1.237 seconds
2018-11-27 05:05:18 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:05:38 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:05:48 [DEBUG   ] Got command RPT from MCS: ref #640478
2018-11-27 05:05:48 [DEBUG   ] summary = ERROR
2018-11-27 05:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640478  13 58449 18348921 A  ERRORERROR'
2018-11-27 05:05:48 [DEBUG   ] Got command RPT from MCS: ref #640479
2018-11-27 05:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640479  77 58449 18348922 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:05:58 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:06:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:06:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 05:06:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:06:03 [DEBUG   ] Finished updating board flags in 1.170 seconds
2018-11-27 05:06:18 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:06:38 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:06:58 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:07:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:07:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:07:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:07:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:07:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:07:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:07:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:07:04 [DEBUG   ] Finished updating board flags in 1.575 seconds
2018-11-27 05:07:18 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:07:38 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:07:57 [DEBUG   ] Got command RPT from MCS: ref #640499
2018-11-27 05:07:57 [DEBUG   ] summary = ERROR
2018-11-27 05:07:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640499  13 58449 18477573 A  ERRORERROR'
2018-11-27 05:07:57 [DEBUG   ] Got command RPT from MCS: ref #640500
2018-11-27 05:07:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:07:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640500  77 58449 18477777 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:07:57 [DEBUG   ] Got command RPT from MCS: ref #640501
2018-11-27 05:07:57 [DEBUG   ] summary = ERROR
2018-11-27 05:07:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640501  13 58449 18477982 A  ERRORERROR'
2018-11-27 05:07:58 [DEBUG   ] Got command RPT from MCS: ref #640502
2018-11-27 05:07:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640502  77 58449 18478174 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:07:58 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:07:58 [DEBUG   ] Got command RPT from MCS: ref #640503
2018-11-27 05:07:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640503  12 58449 18478377 A  ERRORB˜™š'
2018-11-27 05:08:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:08:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:08:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:08:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:08:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:08:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:08:04 [DEBUG   ] Finished updating board flags in 1.461 seconds
2018-11-27 05:08:18 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 05:08:38 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:08:58 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:09:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 05:09:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:09:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:09:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:09:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:09:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:09:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:09:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:09:04 [DEBUG   ] Finished updating board flags in 1.333 seconds
2018-11-27 05:09:18 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:09:38 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:09:58 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:10:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:10:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:10:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:10:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:10:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:10:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:10:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:10:04 [DEBUG   ] Finished updating board flags in 1.215 seconds
2018-11-27 05:10:18 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:10:38 [DEBUG   ] Finished updating ant stats in 0.374 seconds
2018-11-27 05:10:48 [DEBUG   ] Got command RPT from MCS: ref #640508
2018-11-27 05:10:48 [DEBUG   ] summary = ERROR
2018-11-27 05:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640508  13 58449 18648921 A  ERRORERROR'
2018-11-27 05:10:48 [DEBUG   ] Got command RPT from MCS: ref #640509
2018-11-27 05:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640509  77 58449 18648922 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:10:59 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:11:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:11:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:11:04 [DEBUG   ] Finished updating board flags in 1.208 seconds
2018-11-27 05:11:19 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:11:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:11:59 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:12:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:12:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:12:04 [DEBUG   ] Finished updating board flags in 1.182 seconds
2018-11-27 05:12:19 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:12:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:12:57 [DEBUG   ] Got command RPT from MCS: ref #640527
2018-11-27 05:12:57 [DEBUG   ] summary = ERROR
2018-11-27 05:12:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640527  13 58449 18777653 A  ERRORERROR'
2018-11-27 05:12:57 [DEBUG   ] Got command RPT from MCS: ref #640528
2018-11-27 05:12:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:12:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640528  77 58449 18777857 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:12:58 [DEBUG   ] Got command RPT from MCS: ref #640529
2018-11-27 05:12:58 [DEBUG   ] summary = ERROR
2018-11-27 05:12:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640529  13 58449 18778067 A  ERRORERROR'
2018-11-27 05:12:58 [DEBUG   ] Got command RPT from MCS: ref #640531
2018-11-27 05:12:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:12:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640531  77 58449 18778262 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:12:58 [DEBUG   ] Got command RPT from MCS: ref #640532
2018-11-27 05:12:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:12:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640532  12 58449 18778465 A  ERRORB˜™š'
2018-11-27 05:12:59 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 05:13:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:13:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:13:04 [DEBUG   ] Finished updating board flags in 1.012 seconds
2018-11-27 05:13:19 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:13:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:13:59 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:14:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:14:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:14:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:14:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:14:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:14:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:14:04 [DEBUG   ] Finished updating board flags in 1.530 seconds
2018-11-27 05:14:19 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:14:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:14:59 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 05:15:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:15:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:15:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:15:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:15:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:15:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:15:05 [DEBUG   ] Finished updating board flags in 1.941 seconds
2018-11-27 05:15:19 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 05:15:39 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 05:15:48 [DEBUG   ] Got command RPT from MCS: ref #640541
2018-11-27 05:15:48 [DEBUG   ] summary = ERROR
2018-11-27 05:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640541  13 58449 18948917 A  ERRORERROR'
2018-11-27 05:15:48 [DEBUG   ] Got command RPT from MCS: ref #640542
2018-11-27 05:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640542  77 58449 18948918 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:15:59 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 05:16:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:16:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:16:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:16:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:16:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:16:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:16:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:16:04 [DEBUG   ] Finished updating board flags in 1.159 seconds
2018-11-27 05:16:19 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 05:16:39 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 05:16:59 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 05:17:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:17:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:17:04 [DEBUG   ] Finished updating board flags in 1.019 seconds
2018-11-27 05:17:19 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 05:17:39 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 05:17:57 [DEBUG   ] Got command RPT from MCS: ref #640562
2018-11-27 05:17:57 [DEBUG   ] summary = ERROR
2018-11-27 05:17:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640562  13 58449 19077741 A  ERRORERROR'
2018-11-27 05:17:57 [DEBUG   ] Got command RPT from MCS: ref #640563
2018-11-27 05:17:57 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:17:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640563  77 58449 19077945 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:17:58 [DEBUG   ] Got command RPT from MCS: ref #640564
2018-11-27 05:17:58 [DEBUG   ] summary = ERROR
2018-11-27 05:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640564  13 58449 19078149 A  ERRORERROR'
2018-11-27 05:17:58 [DEBUG   ] Got command RPT from MCS: ref #640565
2018-11-27 05:17:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640565  77 58449 19078342 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:17:58 [DEBUG   ] Got command RPT from MCS: ref #640566
2018-11-27 05:17:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640566  12 58449 19078549 A  ERRORB˜™š'
2018-11-27 05:17:59 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 05:18:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:18:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:18:04 [DEBUG   ] Finished updating board flags in 1.021 seconds
2018-11-27 05:18:19 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 05:18:39 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 05:18:59 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 05:19:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 05:19:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:19:04 [DEBUG   ] Finished updating board flags in 0.961 seconds
2018-11-27 05:19:19 [DEBUG   ] Finished updating ant stats in 0.794 seconds
2018-11-27 05:19:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:19:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:20:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:20:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:20:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:20:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:20:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:20:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:20:05 [DEBUG   ] Finished updating board flags in 1.374 seconds
2018-11-27 05:20:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:20:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:20:48 [DEBUG   ] Got command RPT from MCS: ref #640571
2018-11-27 05:20:48 [DEBUG   ] summary = ERROR
2018-11-27 05:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640571  13 58449 19248917 A  ERRORERROR'
2018-11-27 05:20:48 [DEBUG   ] Got command RPT from MCS: ref #640572
2018-11-27 05:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640572  77 58449 19248918 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:20:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:21:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:21:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:21:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:21:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:21:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:21:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:21:05 [DEBUG   ] Finished updating board flags in 1.188 seconds
2018-11-27 05:21:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:21:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:21:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:22:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 05:22:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:22:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:22:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:22:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:22:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:22:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:22:05 [DEBUG   ] Finished updating board flags in 1.081 seconds
2018-11-27 05:22:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:22:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:22:57 [DEBUG   ] Got command RPT from MCS: ref #640594
2018-11-27 05:22:57 [DEBUG   ] summary = ERROR
2018-11-27 05:22:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640594  13 58449 19377829 A  ERRORERROR'
2018-11-27 05:22:58 [DEBUG   ] Got command RPT from MCS: ref #640595
2018-11-27 05:22:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:22:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640595  77 58449 19378033 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:22:58 [DEBUG   ] Got command RPT from MCS: ref #640596
2018-11-27 05:22:58 [DEBUG   ] summary = ERROR
2018-11-27 05:22:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640596  13 58449 19378237 A  ERRORERROR'
2018-11-27 05:22:58 [DEBUG   ] Got command RPT from MCS: ref #640598
2018-11-27 05:22:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:22:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640598  77 58449 19378429 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:22:58 [DEBUG   ] Got command RPT from MCS: ref #640600
2018-11-27 05:22:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:22:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640600  12 58449 19378633 A  ERRORB˜™š'
2018-11-27 05:22:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:23:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 05:23:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:23:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 05:23:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:23:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:23:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:23:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:23:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:23:06 [DEBUG   ] Finished updating board flags in 2.002 seconds
2018-11-27 05:23:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:23:39 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:23:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:24:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:24:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:24:05 [DEBUG   ] Finished updating board flags in 1.382 seconds
2018-11-27 05:24:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:24:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:24:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:25:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 05:25:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:25:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:25:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:25:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:25:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:25:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:25:06 [DEBUG   ] Finished updating board flags in 1.764 seconds
2018-11-27 05:25:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:25:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:25:48 [DEBUG   ] Got command RPT from MCS: ref #640615
2018-11-27 05:25:48 [DEBUG   ] summary = ERROR
2018-11-27 05:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640615  13 58449 19548929 A  ERRORERROR'
2018-11-27 05:25:48 [DEBUG   ] Got command RPT from MCS: ref #640616
2018-11-27 05:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640616  77 58449 19548930 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:25:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:26:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:26:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:26:05 [DEBUG   ] Finished updating board flags in 1.028 seconds
2018-11-27 05:26:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:26:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:26:59 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:27:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 05:27:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:27:05 [DEBUG   ] Finished updating board flags in 1.028 seconds
2018-11-27 05:27:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:27:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:27:57 [DEBUG   ] Got command RPT from MCS: ref #640637
2018-11-27 05:27:57 [DEBUG   ] summary = ERROR
2018-11-27 05:27:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640637  13 58449 19677917 A  ERRORERROR'
2018-11-27 05:27:58 [DEBUG   ] Got command RPT from MCS: ref #640638
2018-11-27 05:27:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640638  77 58449 19678121 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:27:58 [DEBUG   ] Got command RPT from MCS: ref #640639
2018-11-27 05:27:58 [DEBUG   ] summary = ERROR
2018-11-27 05:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640639  13 58449 19678313 A  ERRORERROR'
2018-11-27 05:27:58 [DEBUG   ] Got command RPT from MCS: ref #640640
2018-11-27 05:27:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640640  77 58449 19678517 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:27:58 [DEBUG   ] Got command RPT from MCS: ref #640641
2018-11-27 05:27:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640641  12 58449 19678717 A  ERRORB˜™š'
2018-11-27 05:27:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:28:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:28:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:28:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:28:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:28:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:28:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:28:06 [DEBUG   ] Finished updating board flags in 1.600 seconds
2018-11-27 05:28:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:28:39 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:28:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:29:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 05:29:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:29:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:29:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:29:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:29:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:29:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:29:06 [DEBUG   ] Finished updating board flags in 1.420 seconds
2018-11-27 05:29:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:29:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:29:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:30:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 05:30:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:30:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:30:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:30:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:30:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:30:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:30:06 [DEBUG   ] Finished updating board flags in 1.320 seconds
2018-11-27 05:30:19 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:30:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:30:48 [DEBUG   ] Got command RPT from MCS: ref #640646
2018-11-27 05:30:48 [DEBUG   ] summary = ERROR
2018-11-27 05:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640646  13 58449 19848929 A  ERRORERROR'
2018-11-27 05:30:48 [DEBUG   ] Got command RPT from MCS: ref #640647
2018-11-27 05:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640647  77 58449 19848930 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:30:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:31:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:31:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:31:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:31:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:31:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:31:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:31:06 [DEBUG   ] Finished updating board flags in 1.215 seconds
2018-11-27 05:31:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:31:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:31:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:32:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:32:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:32:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:32:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:32:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:32:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:32:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:32:06 [DEBUG   ] Finished updating board flags in 1.208 seconds
2018-11-27 05:32:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:32:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:32:58 [DEBUG   ] Got command RPT from MCS: ref #640669
2018-11-27 05:32:58 [DEBUG   ] summary = ERROR
2018-11-27 05:32:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640669  13 58449 19978005 A  ERRORERROR'
2018-11-27 05:32:58 [DEBUG   ] Got command RPT from MCS: ref #640670
2018-11-27 05:32:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:32:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640670  77 58449 19978198 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:32:58 [DEBUG   ] Got command RPT from MCS: ref #640671
2018-11-27 05:32:58 [DEBUG   ] summary = ERROR
2018-11-27 05:32:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640671  13 58449 19978401 A  ERRORERROR'
2018-11-27 05:32:58 [DEBUG   ] Got command RPT from MCS: ref #640672
2018-11-27 05:32:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:32:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640672  77 58449 19978605 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:32:58 [DEBUG   ] Got command RPT from MCS: ref #640674
2018-11-27 05:32:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:32:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640674  12 58449 19978809 A  ERRORB—™š'
2018-11-27 05:32:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:33:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:33:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:33:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:33:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:33:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:33:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:33:06 [DEBUG   ] Finished updating board flags in 1.199 seconds
2018-11-27 05:33:19 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:33:39 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:33:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:34:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 05:34:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:34:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:34:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:34:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:34:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:34:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:34:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:34:06 [DEBUG   ] Finished updating board flags in 1.030 seconds
2018-11-27 05:34:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:34:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:34:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:35:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:35:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:35:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:35:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:35:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:35:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:35:07 [DEBUG   ] Finished updating board flags in 2.058 seconds
2018-11-27 05:35:19 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:35:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:35:48 [DEBUG   ] Got command RPT from MCS: ref #640683
2018-11-27 05:35:48 [DEBUG   ] summary = ERROR
2018-11-27 05:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640683  13 58449 20148925 A  ERRORERROR'
2018-11-27 05:35:48 [DEBUG   ] Got command RPT from MCS: ref #640684
2018-11-27 05:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640684  77 58449 20148926 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:35:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:36:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:36:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:36:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:36:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:36:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:36:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:36:06 [DEBUG   ] Finished updating board flags in 1.352 seconds
2018-11-27 05:36:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:36:39 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:36:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:37:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 05:37:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:37:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:37:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:37:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:37:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:37:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:37:06 [DEBUG   ] Finished updating board flags in 1.239 seconds
2018-11-27 05:37:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:37:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:37:58 [DEBUG   ] Got command RPT from MCS: ref #640704
2018-11-27 05:37:58 [DEBUG   ] summary = ERROR
2018-11-27 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640704  13 58449 20278089 A  ERRORERROR'
2018-11-27 05:37:58 [DEBUG   ] Got command RPT from MCS: ref #640705
2018-11-27 05:37:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640705  77 58449 20278293 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:37:58 [DEBUG   ] Got command RPT from MCS: ref #640706
2018-11-27 05:37:58 [DEBUG   ] summary = ERROR
2018-11-27 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640706  13 58449 20278485 A  ERRORERROR'
2018-11-27 05:37:58 [DEBUG   ] Got command RPT from MCS: ref #640707
2018-11-27 05:37:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640707  77 58449 20278690 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:37:58 [DEBUG   ] Got command RPT from MCS: ref #640708
2018-11-27 05:37:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:37:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640708  12 58449 20278893 A  ERRORB˜™š'
2018-11-27 05:37:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:38:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:38:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:38:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:38:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:38:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:38:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:38:06 [DEBUG   ] Finished updating board flags in 1.169 seconds
2018-11-27 05:38:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:38:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:38:59 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:39:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:39:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:39:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:39:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:39:06 [WARNING ] (13, 1, 'bot')
2018-11-27 05:39:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:39:06 [DEBUG   ] Finished updating board flags in 1.028 seconds
2018-11-27 05:39:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:39:39 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:39:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:40:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:40:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:40:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:40:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:40:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:40:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:40:07 [DEBUG   ] Finished updating board flags in 1.598 seconds
2018-11-27 05:40:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:40:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:40:48 [DEBUG   ] Got command RPT from MCS: ref #640713
2018-11-27 05:40:48 [DEBUG   ] summary = ERROR
2018-11-27 05:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640713  13 58449 20448933 A  ERRORERROR'
2018-11-27 05:40:48 [DEBUG   ] Got command RPT from MCS: ref #640714
2018-11-27 05:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640714  77 58449 20448934 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:40:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:41:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:41:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:41:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:41:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:41:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:41:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:41:07 [DEBUG   ] Finished updating board flags in 1.345 seconds
2018-11-27 05:41:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:41:39 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:41:59 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 05:42:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:42:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:42:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:42:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:42:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:42:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:42:07 [DEBUG   ] Finished updating board flags in 1.195 seconds
2018-11-27 05:42:19 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:42:39 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 05:42:58 [DEBUG   ] Got command RPT from MCS: ref #640732
2018-11-27 05:42:58 [DEBUG   ] summary = ERROR
2018-11-27 05:42:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640732  13 58449 20578173 A  ERRORERROR'
2018-11-27 05:42:58 [DEBUG   ] Got command RPT from MCS: ref #640733
2018-11-27 05:42:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:42:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640733  77 58449 20578377 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:42:58 [DEBUG   ] Got command RPT from MCS: ref #640734
2018-11-27 05:42:58 [DEBUG   ] summary = ERROR
2018-11-27 05:42:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640734  13 58449 20578581 A  ERRORERROR'
2018-11-27 05:42:58 [DEBUG   ] Got command RPT from MCS: ref #640736
2018-11-27 05:42:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:42:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640736  77 58449 20578773 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:42:58 [DEBUG   ] Got command RPT from MCS: ref #640737
2018-11-27 05:42:58 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:42:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640737  12 58449 20578977 A  ERRORB—™š'
2018-11-27 05:42:59 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 05:43:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 05:43:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:43:07 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:43:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:43:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:43:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:43:07 [WARNING ] (13, 1, 'bot')
2018-11-27 05:43:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:43:07 [DEBUG   ] Finished updating board flags in 0.991 seconds
2018-11-27 05:43:19 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:43:39 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:43:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:44:00 [DEBUG   ] Got command RPT from MCS: ref #640746
2018-11-27 05:44:00 [DEBUG   ] summary = ERROR
2018-11-27 05:44:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640746  13 58449 20640861 A  ERRORERROR'
2018-11-27 05:44:00 [DEBUG   ] Got command RPT from MCS: ref #640747
2018-11-27 05:44:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:44:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640747  77 58449 20640862 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:44:05 [DEBUG   ] Got command INI from MCS: ref #640748
2018-11-27 05:44:05 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2018-11-27 05:44:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   640748   8 58449 20645870 ABOOTING'
2018-11-27 05:44:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:44:08 [WARNING ] (13, 1, 'bot')
2018-11-27 05:44:08 [WARNING ] (13, 1, 'bot')
2018-11-27 05:44:08 [WARNING ] (13, 1, 'bot')
2018-11-27 05:44:08 [WARNING ] (13, 1, 'bot')
2018-11-27 05:44:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:44:08 [DEBUG   ] Finished updating board flags in 1.822 seconds
2018-11-27 05:44:26 [INFO    ] -------------------
2018-11-27 05:44:26 [INFO    ]  LWA board status
2018-11-27 05:44:26 [INFO    ] -------------------
2018-11-27 05:44:26 [INFO    ] DP 1: 2 lwa27
2018-11-27 05:44:26 [INFO    ] DP 1: 3 lwa24
2018-11-27 05:44:26 [INFO    ] DP 1: 4 lwa23
2018-11-27 05:44:26 [INFO    ] DP 1: 5 lwa18
2018-11-27 05:44:26 [INFO    ] DP 1: 6 lwa10
2018-11-27 05:44:26 [INFO    ] DP 1: 7 lwa32
2018-11-27 05:44:26 [INFO    ] DP 1: 8 lwa11
2018-11-27 05:44:26 [INFO    ] DP 1: 9 lwa30
2018-11-27 05:44:26 [INFO    ] DP 1: 10 lwa12
2018-11-27 05:44:26 [INFO    ] DP 1: 11 lwa8
2018-11-27 05:44:26 [INFO    ] DP 1: 12 lwa13
2018-11-27 05:44:26 [INFO    ] DP 1: 13 lwa26
2018-11-27 05:44:26 [INFO    ] DP 1: 14 lwa14
2018-11-27 05:44:26 [INFO    ] DP 1: 16 lwa22
2018-11-27 05:44:26 [INFO    ] DP 1: 17 lwa20
2018-11-27 05:44:26 [INFO    ] DP 1: 18 lwa1
2018-11-27 05:44:26 [INFO    ] DP 1: 19 lwa5
2018-11-27 05:44:26 [INFO    ] DP 1: 20 lwa25
2018-11-27 05:44:26 [INFO    ] DP 1: 21 lwa35
2018-11-27 05:44:26 [INFO    ] DP 1: 22 lwa19
2018-11-27 05:44:26 [INFO    ] DP 1: 23 lwa21
2018-11-27 05:44:26 [INFO    ] DP 1: 24 lwa6
2018-11-27 05:44:26 [INFO    ] DP 1: 25 lwa7
2018-11-27 05:44:26 [INFO    ] DP 1: 26 lwa33
2018-11-27 05:44:26 [INFO    ] DP 1: 27 lwa15
2018-11-27 05:44:26 [INFO    ] DP 1: 28 lwa28
2018-11-27 05:44:26 [INFO    ] DP 2: 1 lwa16
2018-11-27 05:44:26 [INFO    ] DP 2: 15 lwa34
2018-11-27 05:44:26 [INFO    ] Total of 28 boards
2018-11-27 05:44:26 [INFO    ] Total of 260 stands
2018-11-27 05:44:26 [INFO    ] -------------------
2018-11-27 05:44:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2380
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2381
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2382
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2383
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2384
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2385
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2386
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2387
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2388
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2389
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2390
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2391
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2392
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2393
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2394
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2396
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2400
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2401
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2402
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2403
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2404
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2405
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2406
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2407
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2408
2018-11-27 05:44:33 [DEBUG   ] 2018-11-27 05:44:33 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2409
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2380 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2381 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2382 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2383 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2384 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2385 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2386 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2387 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2388 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2389 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2390 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2391 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2392 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2393 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2394 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2396 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2400 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2401 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2402 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2403 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2404 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2405 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2406 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2407 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2408 exited with status 0
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2409 exited with status 0
2018-11-27 05:44:45 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] spawn_process: at 2018-11-27 05:44:45 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 2438
2018-11-27 05:44:45 [DEBUG   ] 2018-11-27 05:44:45 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 2439
2018-11-27 05:44:51 [DEBUG   ] 2018-11-27 05:44:51 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 2438 exited with status 0
2018-11-27 05:44:51 [DEBUG   ] 2018-11-27 05:44:51 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 2439 exited with status 0
2018-11-27 05:44:51 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2018-11-27 05:44:51 [DEBUG   ] spawn_process: at 2018-11-27 05:44:51 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2018-11-27 05:44:51 [DEBUG   ] spawn_process: at 2018-11-27 05:44:51 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2018-11-27 05:44:51 [INFO    ] Setting Beam 1 xbars
2018-11-27 05:44:51 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2018-11-27 05:44:51 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2018-11-27 05:44:51 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2018-11-27 05:44:51 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2018-11-27 05:44:51 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2018-11-27 05:44:52 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2018-11-27 05:44:52 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2018-11-27 05:44:52 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2018-11-27 05:44:52 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2018-11-27 05:44:52 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2018-11-27 05:44:52 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2018-11-27 05:44:52 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2018-11-27 05:44:52 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2018-11-27 05:44:52 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2018-11-27 05:44:53 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2018-11-27 05:44:53 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2018-11-27 05:44:53 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2018-11-27 05:44:53 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2018-11-27 05:44:53 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2018-11-27 05:44:53 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2018-11-27 05:44:53 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2018-11-27 05:44:53 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2018-11-27 05:44:53 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2018-11-27 05:44:53 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2018-11-27 05:44:54 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2018-11-27 05:44:54 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2018-11-27 05:44:54 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2018-11-27 05:44:54 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2018-11-27 05:44:54 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2018-11-27 05:44:54 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2018-11-27 05:44:54 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2018-11-27 05:44:54 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2018-11-27 05:44:54 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2018-11-27 05:44:55 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2018-11-27 05:44:55 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2018-11-27 05:44:55 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2018-11-27 05:44:55 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2018-11-27 05:44:55 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2018-11-27 05:44:55 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2018-11-27 05:44:55 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2018-11-27 05:44:55 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2018-11-27 05:44:55 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2018-11-27 05:44:56 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2018-11-27 05:44:56 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2018-11-27 05:44:56 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2018-11-27 05:44:56 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2018-11-27 05:44:56 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2018-11-27 05:44:56 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2018-11-27 05:44:56 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2018-11-27 05:44:56 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2018-11-27 05:44:56 [INFO    ] Setting Beam 2 xbars
2018-11-27 05:44:56 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2018-11-27 05:44:57 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2018-11-27 05:44:57 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2018-11-27 05:44:57 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2018-11-27 05:44:57 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2018-11-27 05:44:57 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2018-11-27 05:44:57 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2018-11-27 05:44:57 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2018-11-27 05:44:57 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2018-11-27 05:44:57 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2018-11-27 05:44:57 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2018-11-27 05:44:58 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2018-11-27 05:44:58 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2018-11-27 05:44:58 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2018-11-27 05:44:58 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2018-11-27 05:44:58 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2018-11-27 05:44:58 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2018-11-27 05:44:58 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2018-11-27 05:44:58 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2018-11-27 05:44:58 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2018-11-27 05:44:59 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2018-11-27 05:44:59 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2018-11-27 05:44:59 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2018-11-27 05:44:59 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2018-11-27 05:44:59 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2018-11-27 05:44:59 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2018-11-27 05:44:59 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2018-11-27 05:44:59 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2018-11-27 05:44:59 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2018-11-27 05:45:00 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2018-11-27 05:45:00 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2018-11-27 05:45:00 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2018-11-27 05:45:00 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2018-11-27 05:45:00 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2018-11-27 05:45:00 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2018-11-27 05:45:00 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2018-11-27 05:45:00 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2018-11-27 05:45:00 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2018-11-27 05:45:01 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2018-11-27 05:45:01 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2018-11-27 05:45:01 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2018-11-27 05:45:01 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2018-11-27 05:45:01 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2018-11-27 05:45:01 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2018-11-27 05:45:01 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2018-11-27 05:45:01 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2018-11-27 05:45:01 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2018-11-27 05:45:01 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2018-11-27 05:45:02 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2018-11-27 05:45:02 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2018-11-27 05:45:02 [INFO    ] Setting Beam 3 xbars
2018-11-27 05:45:02 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2018-11-27 05:45:02 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2018-11-27 05:45:02 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2018-11-27 05:45:02 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2018-11-27 05:45:02 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2018-11-27 05:45:02 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2018-11-27 05:45:02 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2018-11-27 05:45:03 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2018-11-27 05:45:03 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2018-11-27 05:45:03 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2018-11-27 05:45:03 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2018-11-27 05:45:03 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2018-11-27 05:45:03 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2018-11-27 05:45:03 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2018-11-27 05:45:03 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2018-11-27 05:45:03 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2018-11-27 05:45:04 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2018-11-27 05:45:04 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2018-11-27 05:45:04 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2018-11-27 05:45:04 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2018-11-27 05:45:04 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2018-11-27 05:45:04 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2018-11-27 05:45:04 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2018-11-27 05:45:04 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2018-11-27 05:45:04 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2018-11-27 05:45:05 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2018-11-27 05:45:05 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2018-11-27 05:45:05 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2018-11-27 05:45:05 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2018-11-27 05:45:05 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2018-11-27 05:45:05 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2018-11-27 05:45:05 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2018-11-27 05:45:05 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2018-11-27 05:45:05 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2018-11-27 05:45:06 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2018-11-27 05:45:06 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2018-11-27 05:45:06 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2018-11-27 05:45:06 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2018-11-27 05:45:06 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2018-11-27 05:45:06 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2018-11-27 05:45:06 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2018-11-27 05:45:06 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2018-11-27 05:45:06 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2018-11-27 05:45:06 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2018-11-27 05:45:07 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2018-11-27 05:45:07 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2018-11-27 05:45:07 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2018-11-27 05:45:07 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2018-11-27 05:45:07 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2018-11-27 05:45:07 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2018-11-27 05:45:07 [INFO    ] Setting Beam 4 xbars
2018-11-27 05:45:07 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2018-11-27 05:45:07 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 05:45:07 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 05:45:08 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 05:45:08 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 05:45:08 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2018-11-27 05:45:08 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2018-11-27 05:45:08 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2018-11-27 05:45:08 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2018-11-27 05:45:08 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2018-11-27 05:45:08 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2018-11-27 05:45:08 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2018-11-27 05:45:09 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2018-11-27 05:45:09 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2018-11-27 05:45:09 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2018-11-27 05:45:09 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2018-11-27 05:45:09 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2018-11-27 05:45:09 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2018-11-27 05:45:09 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2018-11-27 05:45:09 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2018-11-27 05:45:09 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2018-11-27 05:45:10 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2018-11-27 05:45:10 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2018-11-27 05:45:10 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2018-11-27 05:45:10 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2018-11-27 05:45:10 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2018-11-27 05:45:10 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2018-11-27 05:45:10 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2018-11-27 05:45:10 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2018-11-27 05:45:10 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2018-11-27 05:45:11 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2018-11-27 05:45:11 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2018-11-27 05:45:11 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2018-11-27 05:45:11 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2018-11-27 05:45:11 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2018-11-27 05:45:11 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2018-11-27 05:45:11 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2018-11-27 05:45:11 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2018-11-27 05:45:11 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2018-11-27 05:45:12 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2018-11-27 05:45:12 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2018-11-27 05:45:12 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2018-11-27 05:45:12 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2018-11-27 05:45:12 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2018-11-27 05:45:12 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2018-11-27 05:45:12 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2018-11-27 05:45:12 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2018-11-27 05:45:12 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2018-11-27 05:45:13 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2018-11-27 05:45:13 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2018-11-27 05:45:13 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2018-11-27 05:45:13 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2018-11-27 05:45:13 [INFO    ] All xbars set
2018-11-27 05:45:13 [INFO    ] Setting adder chain routing for all beams
2018-11-27 05:45:18 [INFO    ] Setting beam ids
2018-11-27 05:45:28 [INFO    ] Starting beam delay calibration
2018-11-27 05:45:40 [INFO    ] Beam 1 latency is 5734
2018-11-27 05:45:40 [INFO    ] Beam 2 latency is 5730
2018-11-27 05:45:40 [INFO    ] Beam 3 latency is 5736
2018-11-27 05:45:40 [INFO    ] Beam 4 latency is 5733
2018-11-27 05:45:40 [INFO    ] Stop calibration
2018-11-27 05:45:48 [INFO    ] Beam delay calibration successful
2018-11-27 05:45:48 [INFO    ] Calibration passed
2018-11-27 05:45:48 [DEBUG   ] spawn_process: at 2018-11-27 05:45:48 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2018-11-27 05:45:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2018-11-27 05:45:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2018-11-27 05:45:48 [DEBUG   ] spawn_process: at 2018-11-27 05:45:48 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2018-11-27 05:45:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2018-11-27 05:45:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2018-11-27 05:45:48 [DEBUG   ] spawn_process: at 2018-11-27 05:45:48 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2018-11-27 05:45:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2018-11-27 05:45:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2018-11-27 05:45:48 [DEBUG   ] spawn_process: at 2018-11-27 05:45:48 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2018-11-27 05:45:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2018-11-27 05:45:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2018-11-27 05:45:48 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 05:45:48 [DEBUG   ] Got command RPT from MCS: ref #640749
2018-11-27 05:45:48 [DEBUG   ] summary = ERROR
2018-11-27 05:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640749  13 58449 20748933 A  ERRORERROR'
2018-11-27 05:45:48 [DEBUG   ] Got command RPT from MCS: ref #640750
2018-11-27 05:45:48 [DEBUG   ] info = Calibration passed
2018-11-27 05:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640750  26 58449 20748934 A  ERRORCalibration passed'
2018-11-27 05:45:49 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 297 ms (1543297549.299 s) with PID 4127 and time 1543297550
2018-11-27 05:45:49 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 05:45:49 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 384 ms (1543297549.387 s) with PID 4130 and time 1543297550
2018-11-27 05:45:50 [DEBUG   ] Finished updating ant stats in 0.861 seconds
2018-11-27 05:45:50 [INFO    ] Started antennas statistics background thread
2018-11-27 05:45:50 [INFO    ] Started board flags background thread
2018-11-27 05:45:50 [INFO    ] Started BAM background thread
2018-11-27 05:45:50 [INFO    ] Finished the INI process (ref. #640748) in 104.640 s
2018-11-27 05:45:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:45:51 [WARNING ] (13, 1, 'bot')
2018-11-27 05:45:51 [WARNING ] (13, 1, 'bot')
2018-11-27 05:45:51 [WARNING ] (13, 1, 'bot')
2018-11-27 05:45:51 [WARNING ] (13, 1, 'bot')
2018-11-27 05:45:51 [CRITICAL] MGT underflow detected on beam(s) 1, 2, 3, 4
2018-11-27 05:45:51 [DEBUG   ] Finished updating board flags in 1.050 seconds
2018-11-27 05:46:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 05:46:30 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 05:46:50 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 05:46:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:46:51 [WARNING ] (13, 1, 'bot')
2018-11-27 05:46:51 [WARNING ] (13, 1, 'bot')
2018-11-27 05:46:51 [WARNING ] (13, 1, 'bot')
2018-11-27 05:46:51 [WARNING ] (13, 1, 'bot')
2018-11-27 05:46:51 [CRITICAL] MGT underflow detected on beam(s) 1, 2, 3, 4
2018-11-27 05:46:51 [DEBUG   ] Finished updating board flags in 0.987 seconds
2018-11-27 05:47:05 [DEBUG   ] Got command RPT from MCS: ref #640770
2018-11-27 05:47:05 [DEBUG   ] summary = ERROR
2018-11-27 05:47:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640770  13 58449 20825869 A  ERRORERROR'
2018-11-27 05:47:05 [DEBUG   ] Got command RPT from MCS: ref #640771
2018-11-27 05:47:05 [DEBUG   ] info = T_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:47:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640771  98 58449 20825870 A  ERRORT_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:47:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 05:47:10 [DEBUG   ] Got command INI from MCS: ref #640772
2018-11-27 05:47:10 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2018-11-27 05:47:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   640772   8 58449 20830882 ABOOTING'
2018-11-27 05:47:31 [INFO    ] -------------------
2018-11-27 05:47:31 [INFO    ]  LWA board status
2018-11-27 05:47:31 [INFO    ] -------------------
2018-11-27 05:47:31 [INFO    ] DP 1: 2 lwa27
2018-11-27 05:47:31 [INFO    ] DP 1: 3 lwa24
2018-11-27 05:47:31 [INFO    ] DP 1: 4 lwa23
2018-11-27 05:47:31 [INFO    ] DP 1: 5 lwa18
2018-11-27 05:47:31 [INFO    ] DP 1: 6 lwa10
2018-11-27 05:47:31 [INFO    ] DP 1: 7 lwa32
2018-11-27 05:47:31 [INFO    ] DP 1: 8 lwa11
2018-11-27 05:47:31 [INFO    ] DP 1: 9 lwa30
2018-11-27 05:47:31 [INFO    ] DP 1: 10 lwa12
2018-11-27 05:47:31 [INFO    ] DP 1: 11 lwa8
2018-11-27 05:47:31 [INFO    ] DP 1: 12 lwa13
2018-11-27 05:47:31 [INFO    ] DP 1: 13 lwa26
2018-11-27 05:47:31 [INFO    ] DP 1: 14 lwa14
2018-11-27 05:47:31 [INFO    ] DP 1: 16 lwa22
2018-11-27 05:47:31 [INFO    ] DP 1: 17 lwa20
2018-11-27 05:47:31 [INFO    ] DP 1: 18 lwa1
2018-11-27 05:47:31 [INFO    ] DP 1: 19 lwa5
2018-11-27 05:47:31 [INFO    ] DP 1: 20 lwa25
2018-11-27 05:47:31 [INFO    ] DP 1: 21 lwa35
2018-11-27 05:47:31 [INFO    ] DP 1: 22 lwa19
2018-11-27 05:47:31 [INFO    ] DP 1: 23 lwa21
2018-11-27 05:47:31 [INFO    ] DP 1: 24 lwa6
2018-11-27 05:47:31 [INFO    ] DP 1: 25 lwa7
2018-11-27 05:47:31 [INFO    ] DP 1: 26 lwa33
2018-11-27 05:47:31 [INFO    ] DP 1: 27 lwa15
2018-11-27 05:47:31 [INFO    ] DP 1: 28 lwa28
2018-11-27 05:47:31 [INFO    ] DP 2: 1 lwa16
2018-11-27 05:47:31 [INFO    ] DP 2: 15 lwa34
2018-11-27 05:47:31 [INFO    ] Total of 28 boards
2018-11-27 05:47:31 [INFO    ] Total of 260 stands
2018-11-27 05:47:31 [INFO    ] -------------------
2018-11-27 05:47:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4676
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4677
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4678
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4679
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4680
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4681
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4682
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4683
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4684
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4685
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4686
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4687
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4688
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4689
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4690
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4691
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4692
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4693
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4694
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4695
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4696
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4697
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4698
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4699
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4700
2018-11-27 05:47:38 [DEBUG   ] 2018-11-27 05:47:38 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4701
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4676 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4677 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4678 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4679 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4680 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4681 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4682 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4683 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4684 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4685 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4686 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4687 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4688 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4689 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4690 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4691 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4692 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4693 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4694 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4695 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4696 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4697 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4698 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4699 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4700 exited with status 0
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4701 exited with status 0
2018-11-27 05:47:50 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] spawn_process: at 2018-11-27 05:47:50 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 4728
2018-11-27 05:47:50 [DEBUG   ] 2018-11-27 05:47:50 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 4729
2018-11-27 05:47:56 [DEBUG   ] 2018-11-27 05:47:56 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 4728 exited with status 0
2018-11-27 05:47:56 [DEBUG   ] 2018-11-27 05:47:56 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 4729 exited with status 0
2018-11-27 05:47:56 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2018-11-27 05:47:56 [DEBUG   ] spawn_process: at 2018-11-27 05:47:56 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2018-11-27 05:47:56 [DEBUG   ] spawn_process: at 2018-11-27 05:47:56 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2018-11-27 05:47:56 [INFO    ] Setting Beam 1 xbars
2018-11-27 05:47:56 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2018-11-27 05:47:56 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2018-11-27 05:47:56 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2018-11-27 05:47:56 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2018-11-27 05:47:56 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2018-11-27 05:47:57 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2018-11-27 05:47:57 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2018-11-27 05:47:57 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2018-11-27 05:47:57 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2018-11-27 05:47:57 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2018-11-27 05:47:57 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2018-11-27 05:47:57 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2018-11-27 05:47:57 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2018-11-27 05:47:57 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2018-11-27 05:47:58 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2018-11-27 05:47:58 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2018-11-27 05:47:58 [DEBUG   ] Got command RPT from MCS: ref #640773
2018-11-27 05:47:58 [DEBUG   ] summary = BOOTING
2018-11-27 05:47:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640773  15 58449 20878257 ABOOTINGBOOTING'
2018-11-27 05:47:58 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2018-11-27 05:47:58 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2018-11-27 05:47:58 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2018-11-27 05:47:58 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2018-11-27 05:47:58 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2018-11-27 05:47:58 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2018-11-27 05:47:58 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2018-11-27 05:47:59 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2018-11-27 05:47:59 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2018-11-27 05:47:59 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2018-11-27 05:47:59 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2018-11-27 05:47:59 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2018-11-27 05:47:59 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2018-11-27 05:47:59 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2018-11-27 05:47:59 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2018-11-27 05:47:59 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2018-11-27 05:47:59 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2018-11-27 05:48:00 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2018-11-27 05:48:00 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2018-11-27 05:48:00 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2018-11-27 05:48:00 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2018-11-27 05:48:00 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2018-11-27 05:48:00 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2018-11-27 05:48:00 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2018-11-27 05:48:00 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2018-11-27 05:48:00 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2018-11-27 05:48:01 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2018-11-27 05:48:01 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2018-11-27 05:48:01 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2018-11-27 05:48:01 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2018-11-27 05:48:01 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2018-11-27 05:48:01 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2018-11-27 05:48:01 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2018-11-27 05:48:01 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2018-11-27 05:48:01 [INFO    ] Setting Beam 2 xbars
2018-11-27 05:48:01 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2018-11-27 05:48:02 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2018-11-27 05:48:02 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2018-11-27 05:48:02 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2018-11-27 05:48:02 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2018-11-27 05:48:02 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2018-11-27 05:48:02 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2018-11-27 05:48:02 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2018-11-27 05:48:02 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2018-11-27 05:48:02 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2018-11-27 05:48:03 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2018-11-27 05:48:03 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2018-11-27 05:48:03 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2018-11-27 05:48:03 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2018-11-27 05:48:03 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2018-11-27 05:48:03 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2018-11-27 05:48:03 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2018-11-27 05:48:03 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2018-11-27 05:48:03 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2018-11-27 05:48:03 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2018-11-27 05:48:04 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2018-11-27 05:48:04 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2018-11-27 05:48:04 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2018-11-27 05:48:04 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2018-11-27 05:48:04 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2018-11-27 05:48:04 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2018-11-27 05:48:04 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2018-11-27 05:48:04 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2018-11-27 05:48:04 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2018-11-27 05:48:05 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2018-11-27 05:48:05 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2018-11-27 05:48:05 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2018-11-27 05:48:05 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2018-11-27 05:48:05 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2018-11-27 05:48:05 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2018-11-27 05:48:05 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2018-11-27 05:48:05 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2018-11-27 05:48:05 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2018-11-27 05:48:06 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2018-11-27 05:48:06 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2018-11-27 05:48:06 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2018-11-27 05:48:06 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2018-11-27 05:48:06 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2018-11-27 05:48:06 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2018-11-27 05:48:06 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2018-11-27 05:48:06 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2018-11-27 05:48:06 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2018-11-27 05:48:07 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2018-11-27 05:48:07 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2018-11-27 05:48:07 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2018-11-27 05:48:07 [INFO    ] Setting Beam 3 xbars
2018-11-27 05:48:07 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2018-11-27 05:48:07 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2018-11-27 05:48:07 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2018-11-27 05:48:07 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2018-11-27 05:48:07 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2018-11-27 05:48:07 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2018-11-27 05:48:07 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2018-11-27 05:48:08 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2018-11-27 05:48:08 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2018-11-27 05:48:08 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2018-11-27 05:48:08 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2018-11-27 05:48:08 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2018-11-27 05:48:08 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2018-11-27 05:48:08 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2018-11-27 05:48:08 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2018-11-27 05:48:08 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2018-11-27 05:48:09 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2018-11-27 05:48:09 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2018-11-27 05:48:09 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2018-11-27 05:48:09 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2018-11-27 05:48:09 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2018-11-27 05:48:09 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2018-11-27 05:48:09 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2018-11-27 05:48:09 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2018-11-27 05:48:09 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2018-11-27 05:48:10 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2018-11-27 05:48:10 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2018-11-27 05:48:10 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2018-11-27 05:48:10 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2018-11-27 05:48:10 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2018-11-27 05:48:10 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2018-11-27 05:48:10 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2018-11-27 05:48:10 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2018-11-27 05:48:10 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2018-11-27 05:48:11 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2018-11-27 05:48:11 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2018-11-27 05:48:11 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2018-11-27 05:48:11 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2018-11-27 05:48:11 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2018-11-27 05:48:11 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2018-11-27 05:48:11 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2018-11-27 05:48:11 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2018-11-27 05:48:11 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2018-11-27 05:48:12 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2018-11-27 05:48:12 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2018-11-27 05:48:12 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2018-11-27 05:48:12 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2018-11-27 05:48:12 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2018-11-27 05:48:12 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2018-11-27 05:48:12 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2018-11-27 05:48:12 [INFO    ] Setting Beam 4 xbars
2018-11-27 05:48:12 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2018-11-27 05:48:12 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 05:48:12 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 05:48:13 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 05:48:13 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 05:48:13 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2018-11-27 05:48:13 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2018-11-27 05:48:13 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2018-11-27 05:48:13 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2018-11-27 05:48:13 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2018-11-27 05:48:13 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2018-11-27 05:48:13 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2018-11-27 05:48:14 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2018-11-27 05:48:14 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2018-11-27 05:48:14 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2018-11-27 05:48:14 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2018-11-27 05:48:14 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2018-11-27 05:48:14 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2018-11-27 05:48:14 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2018-11-27 05:48:14 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2018-11-27 05:48:14 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2018-11-27 05:48:15 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2018-11-27 05:48:15 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2018-11-27 05:48:15 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2018-11-27 05:48:15 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2018-11-27 05:48:15 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2018-11-27 05:48:15 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2018-11-27 05:48:15 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2018-11-27 05:48:15 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2018-11-27 05:48:15 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2018-11-27 05:48:16 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2018-11-27 05:48:16 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2018-11-27 05:48:16 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2018-11-27 05:48:16 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2018-11-27 05:48:16 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2018-11-27 05:48:16 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2018-11-27 05:48:16 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2018-11-27 05:48:16 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2018-11-27 05:48:16 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2018-11-27 05:48:16 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2018-11-27 05:48:17 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2018-11-27 05:48:17 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2018-11-27 05:48:17 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2018-11-27 05:48:17 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2018-11-27 05:48:17 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2018-11-27 05:48:17 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2018-11-27 05:48:17 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2018-11-27 05:48:17 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2018-11-27 05:48:17 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2018-11-27 05:48:18 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2018-11-27 05:48:18 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2018-11-27 05:48:18 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2018-11-27 05:48:18 [INFO    ] All xbars set
2018-11-27 05:48:18 [INFO    ] Setting adder chain routing for all beams
2018-11-27 05:48:23 [INFO    ] Setting beam ids
2018-11-27 05:48:33 [INFO    ] Starting beam delay calibration
2018-11-27 05:48:45 [INFO    ] Beam 1 latency is 5736
2018-11-27 05:48:45 [INFO    ] Beam 2 latency is 5741
2018-11-27 05:48:45 [INFO    ] Beam 3 latency is 5704
2018-11-27 05:48:45 [INFO    ] Beam 4 latency is 5736
2018-11-27 05:48:45 [INFO    ] Stop calibration
2018-11-27 05:48:53 [INFO    ] Beam delay calibration successful
2018-11-27 05:48:53 [INFO    ] Calibration passed
2018-11-27 05:48:53 [DEBUG   ] spawn_process: at 2018-11-27 05:48:53 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2018-11-27 05:48:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2018-11-27 05:48:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2018-11-27 05:48:53 [DEBUG   ] spawn_process: at 2018-11-27 05:48:53 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2018-11-27 05:48:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2018-11-27 05:48:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2018-11-27 05:48:53 [DEBUG   ] spawn_process: at 2018-11-27 05:48:53 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2018-11-27 05:48:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2018-11-27 05:48:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2018-11-27 05:48:53 [DEBUG   ] spawn_process: at 2018-11-27 05:48:53 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2018-11-27 05:48:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2018-11-27 05:48:54 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2018-11-27 05:48:54 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 05:48:54 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 292 ms (1543297734.294 s) with PID 6310 and time 1543297735
2018-11-27 05:48:54 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 05:48:54 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 381 ms (1543297734.383 s) with PID 6313 and time 1543297735
2018-11-27 05:48:55 [DEBUG   ] Finished updating ant stats in 0.864 seconds
2018-11-27 05:48:55 [INFO    ] Started antennas statistics background thread
2018-11-27 05:48:55 [INFO    ] Started board flags background thread
2018-11-27 05:48:55 [INFO    ] Started BAM background thread
2018-11-27 05:48:55 [INFO    ] Finished the INI process (ref. #640772) in 104.625 s
2018-11-27 05:48:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:48:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:48:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:48:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:48:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:48:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:48:56 [DEBUG   ] Finished updating board flags in 1.056 seconds
2018-11-27 05:49:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 05:49:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 05:49:55 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 05:49:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:49:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:49:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:49:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:49:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:49:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:49:56 [DEBUG   ] Finished updating board flags in 0.987 seconds
2018-11-27 05:50:10 [DEBUG   ] Got command RPT from MCS: ref #640778
2018-11-27 05:50:10 [DEBUG   ] summary = ERROR
2018-11-27 05:50:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640778  13 58449 21010873 A  ERRORERROR'
2018-11-27 05:50:10 [DEBUG   ] Got command RPT from MCS: ref #640779
2018-11-27 05:50:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:50:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640779  77 58449 21010874 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:50:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 05:50:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 05:50:48 [DEBUG   ] Got command RPT from MCS: ref #640780
2018-11-27 05:50:48 [DEBUG   ] summary = ERROR
2018-11-27 05:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640780  13 58449 21048929 A  ERRORERROR'
2018-11-27 05:50:48 [DEBUG   ] Got command RPT from MCS: ref #640781
2018-11-27 05:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640781  77 58449 21048930 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:50:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 05:50:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:50:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:50:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:50:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:50:57 [DEBUG   ] Finished updating board flags in 1.893 seconds
2018-11-27 05:51:15 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 05:51:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:51:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:51:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:51:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:51:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:51:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:51:56 [WARNING ] (13, 1, 'bot')
2018-11-27 05:51:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:51:56 [DEBUG   ] Finished updating board flags in 1.148 seconds
2018-11-27 05:52:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:52:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:52:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:52:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:52:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:52:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:52:57 [DEBUG   ] Finished updating board flags in 1.583 seconds
2018-11-27 05:52:58 [DEBUG   ] Got command RPT from MCS: ref #640803
2018-11-27 05:52:58 [DEBUG   ] summary = ERROR
2018-11-27 05:52:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640803  13 58449 21178345 A  ERRORERROR'
2018-11-27 05:52:58 [DEBUG   ] Got command RPT from MCS: ref #640804
2018-11-27 05:52:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:52:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640804  77 58449 21178549 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:52:58 [DEBUG   ] Got command RPT from MCS: ref #640805
2018-11-27 05:52:58 [DEBUG   ] summary = ERROR
2018-11-27 05:52:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640805  13 58449 21178741 A  ERRORERROR'
2018-11-27 05:52:58 [DEBUG   ] Got command RPT from MCS: ref #640807
2018-11-27 05:52:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:52:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640807  77 58449 21178945 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:52:59 [DEBUG   ] Got command RPT from MCS: ref #640808
2018-11-27 05:52:59 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:52:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640808  12 58449 21179149 A  ERRORB—™š'
2018-11-27 05:53:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:53:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:53:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:53:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 05:53:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:53:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:53:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:53:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:53:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:53:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:53:57 [DEBUG   ] Finished updating board flags in 1.392 seconds
2018-11-27 05:54:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:54:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:54:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:54:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:54:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:54:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:54:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:54:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:54:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:54:57 [DEBUG   ] Finished updating board flags in 1.210 seconds
2018-11-27 05:55:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:55:35 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 05:55:48 [DEBUG   ] Got command RPT from MCS: ref #640817
2018-11-27 05:55:48 [DEBUG   ] summary = ERROR
2018-11-27 05:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640817  13 58449 21348929 A  ERRORERROR'
2018-11-27 05:55:48 [DEBUG   ] Got command RPT from MCS: ref #640818
2018-11-27 05:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640818  77 58449 21348930 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:55:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:55:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:55:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 05:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:55:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:55:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:55:57 [DEBUG   ] Finished updating board flags in 1.212 seconds
2018-11-27 05:56:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:56:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:56:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:56:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 05:56:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:56:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:56:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:56:57 [DEBUG   ] Finished updating board flags in 1.258 seconds
2018-11-27 05:57:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:57:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:57:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:57:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:57:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:57:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:57:57 [DEBUG   ] Finished updating board flags in 1.136 seconds
2018-11-27 05:57:58 [DEBUG   ] Got command RPT from MCS: ref #640834
2018-11-27 05:57:58 [DEBUG   ] summary = ERROR
2018-11-27 05:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640834  13 58449 21478429 A  ERRORERROR'
2018-11-27 05:57:58 [DEBUG   ] Got command RPT from MCS: ref #640835
2018-11-27 05:57:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640835  77 58449 21478633 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:57:58 [DEBUG   ] Got command RPT from MCS: ref #640836
2018-11-27 05:57:58 [DEBUG   ] summary = ERROR
2018-11-27 05:57:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640836  13 58449 21478825 A  ERRORERROR'
2018-11-27 05:57:59 [DEBUG   ] Got command RPT from MCS: ref #640837
2018-11-27 05:57:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 05:57:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640837  77 58449 21479029 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 05:57:59 [DEBUG   ] Got command RPT from MCS: ref #640838
2018-11-27 05:57:59 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 05:57:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640838  12 58449 21479233 A  ERRORB˜™š'
2018-11-27 05:58:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 05:58:35 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 05:58:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:58:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:58:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 05:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:58:57 [WARNING ] (13, 1, 'bot')
2018-11-27 05:58:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:58:57 [DEBUG   ] Finished updating board flags in 0.994 seconds
2018-11-27 05:59:15 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:59:35 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:59:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 05:59:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 05:59:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 05:59:58 [WARNING ] (13, 1, 'bot')
2018-11-27 05:59:58 [WARNING ] (13, 1, 'bot')
2018-11-27 05:59:58 [WARNING ] (13, 1, 'bot')
2018-11-27 05:59:58 [WARNING ] (13, 1, 'bot')
2018-11-27 05:59:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 05:59:58 [DEBUG   ] Finished updating board flags in 1.811 seconds
2018-11-27 06:00:15 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 06:00:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:00:48 [DEBUG   ] Got command RPT from MCS: ref #640843
2018-11-27 06:00:48 [DEBUG   ] summary = ERROR
2018-11-27 06:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640843  13 58449 21648937 A  ERRORERROR'
2018-11-27 06:00:48 [DEBUG   ] Got command RPT from MCS: ref #640844
2018-11-27 06:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640844  77 58449 21648938 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:00:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:00:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:00:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:00:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:00:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:00:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:00:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:00:57 [DEBUG   ] Finished updating board flags in 1.224 seconds
2018-11-27 06:01:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:01:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:01:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:01:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:01:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:01:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:01:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:01:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:01:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:01:57 [DEBUG   ] Finished updating board flags in 1.243 seconds
2018-11-27 06:02:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:02:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:02:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:02:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:02:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:02:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:02:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:02:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:02:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:02:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:02:57 [DEBUG   ] Finished updating board flags in 1.253 seconds
2018-11-27 06:02:58 [DEBUG   ] Got command RPT from MCS: ref #640866
2018-11-27 06:02:58 [DEBUG   ] summary = ERROR
2018-11-27 06:02:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640866  13 58449 21778517 A  ERRORERROR'
2018-11-27 06:02:58 [DEBUG   ] Got command RPT from MCS: ref #640867
2018-11-27 06:02:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:02:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640867  77 58449 21778721 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:02:58 [DEBUG   ] Got command RPT from MCS: ref #640868
2018-11-27 06:02:58 [DEBUG   ] summary = ERROR
2018-11-27 06:02:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640868  13 58449 21778913 A  ERRORERROR'
2018-11-27 06:02:59 [DEBUG   ] Got command RPT from MCS: ref #640870
2018-11-27 06:02:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:02:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640870  77 58449 21779117 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:02:59 [DEBUG   ] Got command RPT from MCS: ref #640871
2018-11-27 06:02:59 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:02:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640871  12 58449 21779321 A  ERRORB˜™š'
2018-11-27 06:03:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:03:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:03:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:03:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 06:03:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:03:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:03:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:03:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:03:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:03:57 [DEBUG   ] Finished updating board flags in 1.239 seconds
2018-11-27 06:04:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:04:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:04:44 [DEBUG   ] Got command DRX from MCS: ref #640884
2018-11-27 06:04:44 [INFO    ] sending drx command to lwa16
2018-11-27 06:04:44 [DEBUG   ] run_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 35100000.0 7 9 0']
2018-11-27 06:04:44 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 06:04:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   640884   8 58449 21884014 A  ERROR'
2018-11-27 06:04:44 [DEBUG   ] Got command DRX from MCS: ref #640885
2018-11-27 06:04:44 [INFO    ] sending drx command to lwa16
2018-11-27 06:04:44 [DEBUG   ] run_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 64500000.0 7 9 0']
2018-11-27 06:04:44 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 06:04:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   640885   8 58449 21884018 A  ERROR'
2018-11-27 06:04:44 [DEBUG   ] Got command DRX from MCS: ref #640886
2018-11-27 06:04:44 [INFO    ] sending drx command to lwa16
2018-11-27 06:04:44 [DEBUG   ] run_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 49800000.0 7 9 0']
2018-11-27 06:04:44 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 06:04:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   640886   8 58449 21884022 A  ERROR'
2018-11-27 06:04:44 [DEBUG   ] Got command DRX from MCS: ref #640887
2018-11-27 06:04:44 [INFO    ] sending drx command to lwa16
2018-11-27 06:04:44 [DEBUG   ] run_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 79200000.0 7 9 0']
2018-11-27 06:04:44 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 06:04:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   640887   8 58449 21884026 A  ERROR'
2018-11-27 06:04:44 [DEBUG   ] Got command BAM from MCS: ref #640888
2018-11-27 06:04:44 [INFO    ] Finished processing BAM request
2018-11-27 06:04:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:04:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640888   8 58449 21884029 A  ERROR'
2018-11-27 06:04:44 [DEBUG   ] Got command BAM from MCS: ref #640889
2018-11-27 06:04:44 [INFO    ] Finished processing BAM request
2018-11-27 06:04:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:04:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640889   8 58449 21884033 A  ERROR'
2018-11-27 06:04:44 [INFO    ] Finished starting DRX
2018-11-27 06:04:44 [INFO    ] Finished starting DRX
2018-11-27 06:04:44 [INFO    ] Finished starting DRX
2018-11-27 06:04:44 [INFO    ] Finished starting DRX
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:04:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:04:44 [DEBUG   ] spawn_process: at 2018-11-27 06:04:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:04:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:04:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:04:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:04:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:04:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:04:58 [DEBUG   ] Finished updating board flags in 1.732 seconds
2018-11-27 06:05:14 [DEBUG   ] Got command BAM from MCS: ref #640892
2018-11-27 06:05:14 [INFO    ] Finished processing BAM request
2018-11-27 06:05:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:05:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640892   8 58449 21914033 A  ERROR'
2018-11-27 06:05:14 [DEBUG   ] Got command BAM from MCS: ref #640893
2018-11-27 06:05:14 [INFO    ] Finished processing BAM request
2018-11-27 06:05:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:05:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640893   8 58449 21914037 A  ERROR'
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:05:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:05:14 [DEBUG   ] spawn_process: at 2018-11-27 06:05:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:05:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:05:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:05:44 [DEBUG   ] Got command BAM from MCS: ref #640894
2018-11-27 06:05:44 [INFO    ] Finished processing BAM request
2018-11-27 06:05:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:05:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640894   8 58449 21944033 A  ERROR'
2018-11-27 06:05:44 [DEBUG   ] Got command BAM from MCS: ref #640895
2018-11-27 06:05:44 [INFO    ] Finished processing BAM request
2018-11-27 06:05:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:05:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640895   8 58449 21944037 A  ERROR'
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:05:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:05:44 [DEBUG   ] spawn_process: at 2018-11-27 06:05:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:05:48 [DEBUG   ] Got command RPT from MCS: ref #640896
2018-11-27 06:05:48 [DEBUG   ] summary = ERROR
2018-11-27 06:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640896  13 58449 21948933 A  ERRORERROR'
2018-11-27 06:05:48 [DEBUG   ] Got command RPT from MCS: ref #640897
2018-11-27 06:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640897  77 58449 21948934 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:05:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:05:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:05:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:05:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:05:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:05:57 [WARNING ] (13, 1, 'bot')
2018-11-27 06:05:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:05:57 [DEBUG   ] Finished updating board flags in 0.951 seconds
2018-11-27 06:06:14 [DEBUG   ] Got command BAM from MCS: ref #640914
2018-11-27 06:06:14 [INFO    ] Finished processing BAM request
2018-11-27 06:06:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:06:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640914   8 58449 21974033 A  ERROR'
2018-11-27 06:06:14 [DEBUG   ] Got command BAM from MCS: ref #640913
2018-11-27 06:06:14 [INFO    ] Finished processing BAM request
2018-11-27 06:06:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:06:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640913   8 58449 21974037 A  ERROR'
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:06:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:06:14 [DEBUG   ] spawn_process: at 2018-11-27 06:06:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:06:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:06:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:06:44 [DEBUG   ] Got command BAM from MCS: ref #640916
2018-11-27 06:06:44 [INFO    ] Finished processing BAM request
2018-11-27 06:06:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:06:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640916   8 58449 22004025 A  ERROR'
2018-11-27 06:06:44 [DEBUG   ] Got command BAM from MCS: ref #640915
2018-11-27 06:06:44 [INFO    ] Finished processing BAM request
2018-11-27 06:06:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:06:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640915   8 58449 22004029 A  ERROR'
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:06:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:06:44 [DEBUG   ] spawn_process: at 2018-11-27 06:06:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:06:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:06:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 06:06:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:06:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 06:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:06:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:06:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:06:58 [DEBUG   ] Finished updating board flags in 1.364 seconds
2018-11-27 06:07:14 [DEBUG   ] Got command BAM from MCS: ref #640921
2018-11-27 06:07:14 [INFO    ] Finished processing BAM request
2018-11-27 06:07:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:07:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640921   8 58449 22034025 A  ERROR'
2018-11-27 06:07:14 [DEBUG   ] Got command BAM from MCS: ref #640922
2018-11-27 06:07:14 [INFO    ] Finished processing BAM request
2018-11-27 06:07:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:07:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640922   8 58449 22034029 A  ERROR'
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:07:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:07:14 [DEBUG   ] spawn_process: at 2018-11-27 06:07:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:07:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:07:36 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 06:07:44 [DEBUG   ] Got command BAM from MCS: ref #640923
2018-11-27 06:07:44 [INFO    ] Finished processing BAM request
2018-11-27 06:07:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:07:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640923   8 58449 22064025 A  ERROR'
2018-11-27 06:07:44 [DEBUG   ] Got command BAM from MCS: ref #640924
2018-11-27 06:07:44 [INFO    ] Finished processing BAM request
2018-11-27 06:07:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:07:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640924   8 58449 22064029 A  ERROR'
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:07:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:07:44 [DEBUG   ] spawn_process: at 2018-11-27 06:07:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:07:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:07:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:07:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:07:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:07:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:07:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:07:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:07:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:07:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:07:58 [DEBUG   ] Finished updating board flags in 1.208 seconds
2018-11-27 06:07:58 [DEBUG   ] Got command RPT from MCS: ref #640925
2018-11-27 06:07:58 [DEBUG   ] summary = ERROR
2018-11-27 06:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640925  13 58449 22078593 A  ERRORERROR'
2018-11-27 06:07:58 [DEBUG   ] Got command RPT from MCS: ref #640926
2018-11-27 06:07:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:07:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640926  77 58449 22078797 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:07:59 [DEBUG   ] Got command RPT from MCS: ref #640927
2018-11-27 06:07:59 [DEBUG   ] summary = ERROR
2018-11-27 06:07:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640927  13 58449 22079001 A  ERRORERROR'
2018-11-27 06:07:59 [DEBUG   ] Got command RPT from MCS: ref #640928
2018-11-27 06:07:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:07:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640928  77 58449 22079205 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:07:59 [DEBUG   ] Got command RPT from MCS: ref #640929
2018-11-27 06:07:59 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:07:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640929  12 58449 22079397 A  ERRORB ff'
2018-11-27 06:08:14 [DEBUG   ] Got command BAM from MCS: ref #640931
2018-11-27 06:08:14 [INFO    ] Finished processing BAM request
2018-11-27 06:08:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:08:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640931   8 58449 22094029 A  ERROR'
2018-11-27 06:08:14 [DEBUG   ] Got command BAM from MCS: ref #640930
2018-11-27 06:08:14 [INFO    ] Finished processing BAM request
2018-11-27 06:08:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:08:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640930   8 58449 22094033 A  ERROR'
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:08:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:08:14 [DEBUG   ] spawn_process: at 2018-11-27 06:08:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:08:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:08:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:08:44 [DEBUG   ] Got command BAM from MCS: ref #640932
2018-11-27 06:08:44 [INFO    ] Finished processing BAM request
2018-11-27 06:08:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:08:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640932   8 58449 22124025 A  ERROR'
2018-11-27 06:08:44 [DEBUG   ] Got command BAM from MCS: ref #640933
2018-11-27 06:08:44 [INFO    ] Finished processing BAM request
2018-11-27 06:08:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:08:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640933   8 58449 22124029 A  ERROR'
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:08:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:08:44 [DEBUG   ] spawn_process: at 2018-11-27 06:08:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:08:56 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 06:08:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:08:58 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:08:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:08:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:08:58 [DEBUG   ] Finished updating board flags in 1.203 seconds
2018-11-27 06:09:14 [DEBUG   ] Got command BAM from MCS: ref #640934
2018-11-27 06:09:14 [INFO    ] Finished processing BAM request
2018-11-27 06:09:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:09:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640934   8 58449 22154037 A  ERROR'
2018-11-27 06:09:14 [DEBUG   ] Got command BAM from MCS: ref #640935
2018-11-27 06:09:14 [INFO    ] Finished processing BAM request
2018-11-27 06:09:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:09:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640935   8 58449 22154045 A  ERROR'
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:09:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:09:14 [DEBUG   ] spawn_process: at 2018-11-27 06:09:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:09:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:09:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:09:44 [DEBUG   ] Got command BAM from MCS: ref #640937
2018-11-27 06:09:44 [INFO    ] Finished processing BAM request
2018-11-27 06:09:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:09:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640937   8 58449 22184033 A  ERROR'
2018-11-27 06:09:44 [DEBUG   ] Got command BAM from MCS: ref #640936
2018-11-27 06:09:44 [INFO    ] Finished processing BAM request
2018-11-27 06:09:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:09:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640936   8 58449 22184037 A  ERROR'
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:09:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:09:44 [DEBUG   ] spawn_process: at 2018-11-27 06:09:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:09:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:09:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:09:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:09:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:09:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:09:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:09:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:09:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:09:58 [DEBUG   ] Finished updating board flags in 1.203 seconds
2018-11-27 06:10:14 [DEBUG   ] Got command BAM from MCS: ref #640942
2018-11-27 06:10:14 [INFO    ] Finished processing BAM request
2018-11-27 06:10:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:10:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640942   8 58449 22214029 A  ERROR'
2018-11-27 06:10:14 [DEBUG   ] Got command BAM from MCS: ref #640943
2018-11-27 06:10:14 [INFO    ] Finished processing BAM request
2018-11-27 06:10:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:10:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640943   8 58449 22214033 A  ERROR'
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:10:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:10:14 [DEBUG   ] spawn_process: at 2018-11-27 06:10:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:10:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:10:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:10:44 [DEBUG   ] Got command BAM from MCS: ref #640944
2018-11-27 06:10:44 [INFO    ] Finished processing BAM request
2018-11-27 06:10:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:10:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640944   8 58449 22244037 A  ERROR'
2018-11-27 06:10:44 [DEBUG   ] Got command BAM from MCS: ref #640945
2018-11-27 06:10:44 [INFO    ] Finished processing BAM request
2018-11-27 06:10:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:10:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640945   8 58449 22244045 A  ERROR'
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:10:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:10:44 [DEBUG   ] spawn_process: at 2018-11-27 06:10:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:10:48 [DEBUG   ] Got command RPT from MCS: ref #640946
2018-11-27 06:10:48 [DEBUG   ] summary = ERROR
2018-11-27 06:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640946  13 58449 22248941 A  ERRORERROR'
2018-11-27 06:10:48 [DEBUG   ] Got command RPT from MCS: ref #640947
2018-11-27 06:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640947  77 58449 22248942 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:10:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:10:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 06:10:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 06:10:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:10:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:10:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:10:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:10:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:10:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:10:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:10:58 [DEBUG   ] Finished updating board flags in 1.214 seconds
2018-11-27 06:11:14 [DEBUG   ] Got command BAM from MCS: ref #640964
2018-11-27 06:11:14 [INFO    ] Finished processing BAM request
2018-11-27 06:11:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:11:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640964   8 58449 22274033 A  ERROR'
2018-11-27 06:11:14 [DEBUG   ] Got command BAM from MCS: ref #640963
2018-11-27 06:11:14 [INFO    ] Finished processing BAM request
2018-11-27 06:11:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:11:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640963   8 58449 22274037 A  ERROR'
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:11:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:11:14 [DEBUG   ] spawn_process: at 2018-11-27 06:11:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:11:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:11:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:11:44 [DEBUG   ] Got command BAM from MCS: ref #640966
2018-11-27 06:11:44 [INFO    ] Finished processing BAM request
2018-11-27 06:11:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:11:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640966   8 58449 22304029 A  ERROR'
2018-11-27 06:11:44 [DEBUG   ] Got command BAM from MCS: ref #640965
2018-11-27 06:11:44 [INFO    ] Finished processing BAM request
2018-11-27 06:11:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:11:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640965   8 58449 22304033 A  ERROR'
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:11:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:11:44 [DEBUG   ] spawn_process: at 2018-11-27 06:11:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:11:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:11:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:11:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 06:11:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 06:11:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:11:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 06:11:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:11:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:11:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:11:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:11:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:11:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:11:58 [DEBUG   ] Finished updating board flags in 1.200 seconds
2018-11-27 06:12:14 [DEBUG   ] Got command BAM from MCS: ref #640967
2018-11-27 06:12:14 [INFO    ] Finished processing BAM request
2018-11-27 06:12:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:12:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640967   8 58449 22334029 A  ERROR'
2018-11-27 06:12:14 [DEBUG   ] Got command BAM from MCS: ref #640968
2018-11-27 06:12:14 [INFO    ] Finished processing BAM request
2018-11-27 06:12:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:12:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640968   8 58449 22334033 A  ERROR'
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:12:14 [DEBUG   ] spawn_process: at 2018-11-27 06:12:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:12:16 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 06:12:36 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:12:44 [DEBUG   ] Got command BAM from MCS: ref #640972
2018-11-27 06:12:44 [INFO    ] Finished processing BAM request
2018-11-27 06:12:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:12:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640972   8 58449 22364033 A  ERROR'
2018-11-27 06:12:44 [DEBUG   ] Got command BAM from MCS: ref #640971
2018-11-27 06:12:44 [INFO    ] Finished processing BAM request
2018-11-27 06:12:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:12:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640971   8 58449 22364037 A  ERROR'
2018-11-27 06:12:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:12:44 [DEBUG   ] spawn_process: at 2018-11-27 06:12:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:12:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:12:44 [DEBUG   ] spawn_process: at 2018-11-27 06:12:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:12:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:12:44 [DEBUG   ] spawn_process: at 2018-11-27 06:12:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:12:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:12:45 [DEBUG   ] spawn_process: at 2018-11-27 06:12:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:12:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:12:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 06:12:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:12:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:12:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:12:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:12:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:12:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:12:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:12:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:12:58 [DEBUG   ] Finished updating board flags in 0.999 seconds
2018-11-27 06:12:58 [DEBUG   ] Got command RPT from MCS: ref #640973
2018-11-27 06:12:58 [DEBUG   ] summary = ERROR
2018-11-27 06:12:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640973  13 58449 22378681 A  ERRORERROR'
2018-11-27 06:12:58 [DEBUG   ] Got command RPT from MCS: ref #640974
2018-11-27 06:12:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:12:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640974  77 58449 22378885 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:12:59 [DEBUG   ] Got command RPT from MCS: ref #640975
2018-11-27 06:12:59 [DEBUG   ] summary = ERROR
2018-11-27 06:12:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640975  13 58449 22379089 A  ERRORERROR'
2018-11-27 06:12:59 [DEBUG   ] Got command RPT from MCS: ref #640977
2018-11-27 06:12:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:12:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640977  77 58449 22379293 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:12:59 [DEBUG   ] Got command RPT from MCS: ref #640978
2018-11-27 06:12:59 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:12:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640978  12 58449 22379485 A  ERRORB¢ff'
2018-11-27 06:13:14 [DEBUG   ] Got command BAM from MCS: ref #640988
2018-11-27 06:13:14 [INFO    ] Finished processing BAM request
2018-11-27 06:13:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:13:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640988   8 58449 22394033 A  ERROR'
2018-11-27 06:13:14 [DEBUG   ] Got command BAM from MCS: ref #640987
2018-11-27 06:13:14 [INFO    ] Finished processing BAM request
2018-11-27 06:13:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:13:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640987   8 58449 22394037 A  ERROR'
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:13:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:13:14 [DEBUG   ] spawn_process: at 2018-11-27 06:13:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:13:16 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:13:36 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:13:44 [DEBUG   ] Got command BAM from MCS: ref #640990
2018-11-27 06:13:44 [INFO    ] Finished processing BAM request
2018-11-27 06:13:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:13:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640990   8 58449 22424025 A  ERROR'
2018-11-27 06:13:44 [DEBUG   ] Got command BAM from MCS: ref #640989
2018-11-27 06:13:44 [INFO    ] Finished processing BAM request
2018-11-27 06:13:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:13:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640989   8 58449 22424029 A  ERROR'
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:13:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:13:44 [DEBUG   ] spawn_process: at 2018-11-27 06:13:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:13:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:13:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 06:13:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:13:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:13:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 06:13:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:13:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:13:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:13:59 [DEBUG   ] Finished updating board flags in 1.865 seconds
2018-11-27 06:14:14 [DEBUG   ] Got command BAM from MCS: ref #640991
2018-11-27 06:14:14 [INFO    ] Finished processing BAM request
2018-11-27 06:14:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:14:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640991   8 58449 22454033 A  ERROR'
2018-11-27 06:14:14 [DEBUG   ] Got command BAM from MCS: ref #640992
2018-11-27 06:14:14 [INFO    ] Finished processing BAM request
2018-11-27 06:14:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:14:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640992   8 58449 22454037 A  ERROR'
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:14:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:14:14 [DEBUG   ] spawn_process: at 2018-11-27 06:14:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:14:16 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 06:14:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 06:14:44 [DEBUG   ] Got command BAM from MCS: ref #640994
2018-11-27 06:14:44 [INFO    ] Finished processing BAM request
2018-11-27 06:14:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:14:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640994   8 58449 22484029 A  ERROR'
2018-11-27 06:14:44 [DEBUG   ] Got command BAM from MCS: ref #640993
2018-11-27 06:14:44 [INFO    ] Finished processing BAM request
2018-11-27 06:14:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:14:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640993   8 58449 22484033 A  ERROR'
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:14:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:14:44 [DEBUG   ] spawn_process: at 2018-11-27 06:14:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:14:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 06:14:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 06:14:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:14:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:14:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:14:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:14:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:14:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:14:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:14:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:14:58 [DEBUG   ] Finished updating board flags in 1.168 seconds
2018-11-27 06:15:14 [DEBUG   ] Got command BAM from MCS: ref #640996
2018-11-27 06:15:14 [INFO    ] Finished processing BAM request
2018-11-27 06:15:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:15:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640996   8 58449 22514033 A  ERROR'
2018-11-27 06:15:14 [DEBUG   ] Got command BAM from MCS: ref #640995
2018-11-27 06:15:14 [INFO    ] Finished processing BAM request
2018-11-27 06:15:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:15:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640995   8 58449 22514037 A  ERROR'
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:15:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:15:14 [DEBUG   ] spawn_process: at 2018-11-27 06:15:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:15:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 06:15:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 06:15:44 [DEBUG   ] Got command BAM from MCS: ref #640997
2018-11-27 06:15:44 [INFO    ] Finished processing BAM request
2018-11-27 06:15:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:15:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640997   8 58449 22544033 A  ERROR'
2018-11-27 06:15:44 [DEBUG   ] Got command BAM from MCS: ref #640998
2018-11-27 06:15:44 [INFO    ] Finished processing BAM request
2018-11-27 06:15:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:15:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   640998   8 58449 22544037 A  ERROR'
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:15:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:15:44 [DEBUG   ] spawn_process: at 2018-11-27 06:15:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:15:48 [DEBUG   ] Got command RPT from MCS: ref #640999
2018-11-27 06:15:48 [DEBUG   ] summary = ERROR
2018-11-27 06:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   640999  13 58449 22548933 A  ERRORERROR'
2018-11-27 06:15:48 [DEBUG   ] Got command RPT from MCS: ref #641000
2018-11-27 06:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641000  77 58449 22548934 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:15:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 06:15:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:15:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 06:15:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 06:15:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:15:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:15:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:15:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:15:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:15:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:15:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:15:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:15:58 [DEBUG   ] Finished updating board flags in 1.034 seconds
2018-11-27 06:16:14 [DEBUG   ] Got command BAM from MCS: ref #641020
2018-11-27 06:16:14 [INFO    ] Finished processing BAM request
2018-11-27 06:16:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:16:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641020   8 58449 22574033 A  ERROR'
2018-11-27 06:16:14 [DEBUG   ] Got command BAM from MCS: ref #641021
2018-11-27 06:16:14 [INFO    ] Finished processing BAM request
2018-11-27 06:16:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:16:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641021   8 58449 22574037 A  ERROR'
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:16:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:16:14 [DEBUG   ] spawn_process: at 2018-11-27 06:16:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:16:16 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 06:16:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:16:44 [DEBUG   ] Got command BAM from MCS: ref #641022
2018-11-27 06:16:44 [INFO    ] Finished processing BAM request
2018-11-27 06:16:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:16:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641022   8 58449 22604025 A  ERROR'
2018-11-27 06:16:44 [DEBUG   ] Got command BAM from MCS: ref #641023
2018-11-27 06:16:44 [INFO    ] Finished processing BAM request
2018-11-27 06:16:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:16:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641023   8 58449 22604029 A  ERROR'
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:16:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:16:45 [DEBUG   ] spawn_process: at 2018-11-27 06:16:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:16:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:16:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:16:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 06:16:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 06:16:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:16:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:16:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2018-11-27 06:16:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:16:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:16:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:16:58 [WARNING ] (13, 1, 'bot')
2018-11-27 06:16:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:16:58 [DEBUG   ] Finished updating board flags in 0.974 seconds
2018-11-27 06:17:14 [DEBUG   ] Got command BAM from MCS: ref #641025
2018-11-27 06:17:14 [INFO    ] Finished processing BAM request
2018-11-27 06:17:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:17:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641025   8 58449 22634025 A  ERROR'
2018-11-27 06:17:14 [DEBUG   ] Got command BAM from MCS: ref #641024
2018-11-27 06:17:14 [INFO    ] Finished processing BAM request
2018-11-27 06:17:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:17:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641024   8 58449 22634029 A  ERROR'
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:17:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:17:14 [DEBUG   ] spawn_process: at 2018-11-27 06:17:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:17:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:17:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:17:44 [DEBUG   ] Got command BAM from MCS: ref #641027
2018-11-27 06:17:44 [INFO    ] Finished processing BAM request
2018-11-27 06:17:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:17:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641027   8 58449 22664025 A  ERROR'
2018-11-27 06:17:44 [DEBUG   ] Got command BAM from MCS: ref #641026
2018-11-27 06:17:44 [INFO    ] Finished processing BAM request
2018-11-27 06:17:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:17:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641026   8 58449 22664029 A  ERROR'
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:17:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:17:44 [DEBUG   ] spawn_process: at 2018-11-27 06:17:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:17:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:17:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 06:17:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:17:58 [DEBUG   ] Got command RPT from MCS: ref #641028
2018-11-27 06:17:58 [DEBUG   ] summary = ERROR
2018-11-27 06:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641028  13 58449 22678773 A  ERRORERROR'
2018-11-27 06:17:58 [DEBUG   ] Got command RPT from MCS: ref #641029
2018-11-27 06:17:58 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641029  77 58449 22678977 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:17:59 [DEBUG   ] Got command RPT from MCS: ref #641030
2018-11-27 06:17:59 [DEBUG   ] summary = ERROR
2018-11-27 06:17:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641030  13 58449 22679169 A  ERRORERROR'
2018-11-27 06:17:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 06:17:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:17:59 [DEBUG   ] Got command RPT from MCS: ref #641031
2018-11-27 06:17:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:17:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641031  77 58449 22679375 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:17:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:17:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:17:59 [DEBUG   ] Finished updating board flags in 1.370 seconds
2018-11-27 06:17:59 [DEBUG   ] Got command RPT from MCS: ref #641032
2018-11-27 06:17:59 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:17:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641032  12 58449 22679577 A  ERRORB¢ff'
2018-11-27 06:18:14 [DEBUG   ] Got command BAM from MCS: ref #641033
2018-11-27 06:18:14 [INFO    ] Finished processing BAM request
2018-11-27 06:18:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:18:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641033   8 58449 22694029 A  ERROR'
2018-11-27 06:18:14 [DEBUG   ] Got command BAM from MCS: ref #641034
2018-11-27 06:18:14 [INFO    ] Finished processing BAM request
2018-11-27 06:18:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:18:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641034   8 58449 22694033 A  ERROR'
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:18:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:18:14 [DEBUG   ] spawn_process: at 2018-11-27 06:18:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:18:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:18:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:18:44 [DEBUG   ] Got command BAM from MCS: ref #641035
2018-11-27 06:18:44 [INFO    ] Finished processing BAM request
2018-11-27 06:18:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:18:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641035   8 58449 22724029 A  ERROR'
2018-11-27 06:18:44 [DEBUG   ] Got command BAM from MCS: ref #641036
2018-11-27 06:18:44 [INFO    ] Finished processing BAM request
2018-11-27 06:18:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:18:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641036   8 58449 22724033 A  ERROR'
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:18:44 [DEBUG   ] spawn_process: at 2018-11-27 06:18:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:18:56 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 06:18:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:18:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:18:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 06:18:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:18:59 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:18:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:18:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:18:59 [DEBUG   ] Finished updating board flags in 1.207 seconds
2018-11-27 06:19:14 [DEBUG   ] Got command BAM from MCS: ref #641042
2018-11-27 06:19:14 [INFO    ] Finished processing BAM request
2018-11-27 06:19:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:19:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641042   8 58449 22754029 A  ERROR'
2018-11-27 06:19:14 [DEBUG   ] Got command BAM from MCS: ref #641041
2018-11-27 06:19:14 [INFO    ] Finished processing BAM request
2018-11-27 06:19:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:19:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641041   8 58449 22754033 A  ERROR'
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:19:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:19:14 [DEBUG   ] spawn_process: at 2018-11-27 06:19:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:19:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:19:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:19:44 [DEBUG   ] Got command BAM from MCS: ref #641043
2018-11-27 06:19:44 [INFO    ] Finished processing BAM request
2018-11-27 06:19:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:19:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641043   8 58449 22784033 A  ERROR'
2018-11-27 06:19:44 [DEBUG   ] Got command BAM from MCS: ref #641044
2018-11-27 06:19:44 [INFO    ] Finished processing BAM request
2018-11-27 06:19:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:19:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641044   8 58449 22784037 A  ERROR'
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:19:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:19:44 [DEBUG   ] spawn_process: at 2018-11-27 06:19:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:19:56 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 06:19:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 06:19:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 06:19:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:19:59 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:19:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:19:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:19:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:19:59 [DEBUG   ] Finished updating board flags in 1.195 seconds
2018-11-27 06:20:14 [DEBUG   ] Got command BAM from MCS: ref #641046
2018-11-27 06:20:14 [INFO    ] Finished processing BAM request
2018-11-27 06:20:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:20:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641046   8 58449 22814029 A  ERROR'
2018-11-27 06:20:14 [DEBUG   ] Got command BAM from MCS: ref #641045
2018-11-27 06:20:14 [INFO    ] Finished processing BAM request
2018-11-27 06:20:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:20:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641045   8 58449 22814033 A  ERROR'
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:20:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:20:14 [DEBUG   ] spawn_process: at 2018-11-27 06:20:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:20:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:20:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:20:44 [DEBUG   ] Got command BAM from MCS: ref #641047
2018-11-27 06:20:44 [INFO    ] Finished processing BAM request
2018-11-27 06:20:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:20:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641047   8 58449 22844029 A  ERROR'
2018-11-27 06:20:44 [DEBUG   ] Got command BAM from MCS: ref #641048
2018-11-27 06:20:44 [INFO    ] Finished processing BAM request
2018-11-27 06:20:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:20:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641048   8 58449 22844033 A  ERROR'
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:20:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:20:44 [DEBUG   ] spawn_process: at 2018-11-27 06:20:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:20:48 [DEBUG   ] Got command RPT from MCS: ref #641049
2018-11-27 06:20:48 [DEBUG   ] summary = ERROR
2018-11-27 06:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641049  13 58449 22848941 A  ERRORERROR'
2018-11-27 06:20:48 [DEBUG   ] Got command RPT from MCS: ref #641050
2018-11-27 06:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641050  77 58449 22848942 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:20:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:20:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2018-11-27 06:20:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 06:20:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:20:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 06:20:59 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:20:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:20:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:20:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:20:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:20:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:20:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:20:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:20:59 [DEBUG   ] Finished updating board flags in 1.004 seconds
2018-11-27 06:21:14 [DEBUG   ] Got command BAM from MCS: ref #641067
2018-11-27 06:21:14 [INFO    ] Finished processing BAM request
2018-11-27 06:21:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:21:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641067   8 58449 22874033 A  ERROR'
2018-11-27 06:21:14 [DEBUG   ] Got command BAM from MCS: ref #641066
2018-11-27 06:21:14 [INFO    ] Finished processing BAM request
2018-11-27 06:21:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:21:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641066   8 58449 22874037 A  ERROR'
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:21:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:21:15 [DEBUG   ] spawn_process: at 2018-11-27 06:21:15 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:21:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:21:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:21:44 [DEBUG   ] Got command BAM from MCS: ref #641068
2018-11-27 06:21:44 [INFO    ] Finished processing BAM request
2018-11-27 06:21:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:21:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641068   8 58449 22904029 A  ERROR'
2018-11-27 06:21:44 [DEBUG   ] Got command BAM from MCS: ref #641069
2018-11-27 06:21:44 [INFO    ] Finished processing BAM request
2018-11-27 06:21:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:21:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641069   8 58449 22904033 A  ERROR'
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:21:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:21:44 [DEBUG   ] spawn_process: at 2018-11-27 06:21:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:21:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:21:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:21:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:21:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 06:21:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:21:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:21:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:21:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:21:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:21:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:21:59 [DEBUG   ] Finished updating board flags in 1.063 seconds
2018-11-27 06:22:14 [DEBUG   ] Got command BAM from MCS: ref #641075
2018-11-27 06:22:14 [INFO    ] Finished processing BAM request
2018-11-27 06:22:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:22:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641075   8 58449 22934025 A  ERROR'
2018-11-27 06:22:14 [DEBUG   ] Got command BAM from MCS: ref #641074
2018-11-27 06:22:14 [INFO    ] Finished processing BAM request
2018-11-27 06:22:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:22:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641074   8 58449 22934029 A  ERROR'
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:22:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:22:14 [DEBUG   ] spawn_process: at 2018-11-27 06:22:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:22:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:22:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:22:44 [DEBUG   ] Got command BAM from MCS: ref #641078
2018-11-27 06:22:44 [INFO    ] Finished processing BAM request
2018-11-27 06:22:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:22:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641078   8 58449 22964033 A  ERROR'
2018-11-27 06:22:44 [DEBUG   ] Got command BAM from MCS: ref #641079
2018-11-27 06:22:44 [INFO    ] Finished processing BAM request
2018-11-27 06:22:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:22:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641079   8 58449 22964037 A  ERROR'
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:22:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:22:44 [DEBUG   ] spawn_process: at 2018-11-27 06:22:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:22:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:22:58 [DEBUG   ] Got command RPT from MCS: ref #641080
2018-11-27 06:22:58 [DEBUG   ] summary = ERROR
2018-11-27 06:22:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641080  13 58449 22978857 A  ERRORERROR'
2018-11-27 06:22:59 [DEBUG   ] Got command RPT from MCS: ref #641081
2018-11-27 06:22:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641081  77 58449 22979061 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:22:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:22:59 [DEBUG   ] Got command RPT from MCS: ref #641082
2018-11-27 06:22:59 [DEBUG   ] summary = ERROR
2018-11-27 06:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641082  13 58449 22979265 A  ERRORERROR'
2018-11-27 06:22:59 [DEBUG   ] Got command RPT from MCS: ref #641084
2018-11-27 06:22:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641084  77 58449 22979457 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:22:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:22:59 [DEBUG   ] Got command RPT from MCS: ref #641085
2018-11-27 06:22:59 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641085  12 58449 22979661 A  ERRORB›™š'
2018-11-27 06:23:00 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:23:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:23:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:23:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:23:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:23:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:23:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:23:00 [DEBUG   ] Finished updating board flags in 2.011 seconds
2018-11-27 06:23:14 [DEBUG   ] Got command BAM from MCS: ref #641091
2018-11-27 06:23:14 [INFO    ] Finished processing BAM request
2018-11-27 06:23:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:23:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641091   8 58449 22994029 A  ERROR'
2018-11-27 06:23:14 [DEBUG   ] Got command BAM from MCS: ref #641090
2018-11-27 06:23:14 [INFO    ] Finished processing BAM request
2018-11-27 06:23:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:23:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641090   8 58449 22994033 A  ERROR'
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:23:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:23:14 [DEBUG   ] spawn_process: at 2018-11-27 06:23:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:23:16 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 06:23:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:23:44 [DEBUG   ] Got command BAM from MCS: ref #641093
2018-11-27 06:23:44 [INFO    ] Finished processing BAM request
2018-11-27 06:23:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:23:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641093   8 58449 23024025 A  ERROR'
2018-11-27 06:23:44 [DEBUG   ] Got command BAM from MCS: ref #641092
2018-11-27 06:23:44 [INFO    ] Finished processing BAM request
2018-11-27 06:23:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:23:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641092   8 58449 23024029 A  ERROR'
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:23:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:23:44 [DEBUG   ] spawn_process: at 2018-11-27 06:23:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:23:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:23:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:23:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 06:24:00 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:24:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:24:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:24:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:24:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:24:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:24:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:24:00 [DEBUG   ] Finished updating board flags in 1.927 seconds
2018-11-27 06:24:14 [DEBUG   ] Got command BAM from MCS: ref #641094
2018-11-27 06:24:14 [INFO    ] Finished processing BAM request
2018-11-27 06:24:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:24:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641094   8 58449 23054029 A  ERROR'
2018-11-27 06:24:14 [DEBUG   ] Got command BAM from MCS: ref #641095
2018-11-27 06:24:14 [INFO    ] Finished processing BAM request
2018-11-27 06:24:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:24:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641095   8 58449 23054033 A  ERROR'
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:24:14 [DEBUG   ] spawn_process: at 2018-11-27 06:24:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:24:17 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 06:24:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:24:44 [DEBUG   ] Got command BAM from MCS: ref #641097
2018-11-27 06:24:44 [INFO    ] Finished processing BAM request
2018-11-27 06:24:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:24:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641097   8 58449 23084033 A  ERROR'
2018-11-27 06:24:44 [DEBUG   ] Got command BAM from MCS: ref #641096
2018-11-27 06:24:44 [INFO    ] Finished processing BAM request
2018-11-27 06:24:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:24:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641096   8 58449 23084037 A  ERROR'
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:24:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:24:44 [DEBUG   ] spawn_process: at 2018-11-27 06:24:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:24:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:24:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 06:24:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:24:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 06:24:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:24:59 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:24:59 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:24:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:24:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:24:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:24:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:24:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:24:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:24:59 [DEBUG   ] Finished updating board flags in 1.247 seconds
2018-11-27 06:25:14 [DEBUG   ] Got command BAM from MCS: ref #641103
2018-11-27 06:25:14 [INFO    ] Finished processing BAM request
2018-11-27 06:25:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:25:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641103   8 58449 23114026 A  ERROR'
2018-11-27 06:25:14 [DEBUG   ] Got command BAM from MCS: ref #641102
2018-11-27 06:25:14 [INFO    ] Finished processing BAM request
2018-11-27 06:25:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:25:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641102   8 58449 23114030 A  ERROR'
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:25:15 [DEBUG   ] spawn_process: at 2018-11-27 06:25:15 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:25:17 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 06:25:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:25:44 [DEBUG   ] Got command BAM from MCS: ref #641105
2018-11-27 06:25:44 [INFO    ] Finished processing BAM request
2018-11-27 06:25:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:25:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641105   8 58449 23144034 A  ERROR'
2018-11-27 06:25:44 [DEBUG   ] Got command BAM from MCS: ref #641104
2018-11-27 06:25:44 [INFO    ] Finished processing BAM request
2018-11-27 06:25:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:25:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641104   8 58449 23144038 A  ERROR'
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:25:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:25:44 [DEBUG   ] spawn_process: at 2018-11-27 06:25:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:25:48 [DEBUG   ] Got command RPT from MCS: ref #641106
2018-11-27 06:25:48 [DEBUG   ] summary = ERROR
2018-11-27 06:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641106  13 58449 23148934 A  ERRORERROR'
2018-11-27 06:25:48 [DEBUG   ] Got command RPT from MCS: ref #641107
2018-11-27 06:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641107  77 58449 23148934 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:25:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:25:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:25:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2018-11-27 06:25:59 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:25:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:25:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:25:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:25:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:25:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:25:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:25:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:25:59 [DEBUG   ] Finished updating board flags in 1.230 seconds
2018-11-27 06:26:14 [DEBUG   ] Got command BAM from MCS: ref #641123
2018-11-27 06:26:14 [INFO    ] Finished processing BAM request
2018-11-27 06:26:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:26:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641123   8 58449 23174030 A  ERROR'
2018-11-27 06:26:14 [DEBUG   ] Got command BAM from MCS: ref #641124
2018-11-27 06:26:14 [INFO    ] Finished processing BAM request
2018-11-27 06:26:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:26:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641124   8 58449 23174034 A  ERROR'
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:26:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:26:14 [DEBUG   ] spawn_process: at 2018-11-27 06:26:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:26:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:26:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:26:44 [DEBUG   ] Got command BAM from MCS: ref #641125
2018-11-27 06:26:44 [INFO    ] Finished processing BAM request
2018-11-27 06:26:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:26:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641125   8 58449 23204026 A  ERROR'
2018-11-27 06:26:44 [DEBUG   ] Got command BAM from MCS: ref #641126
2018-11-27 06:26:44 [INFO    ] Finished processing BAM request
2018-11-27 06:26:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:26:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641126   8 58449 23204030 A  ERROR'
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:26:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:26:44 [DEBUG   ] spawn_process: at 2018-11-27 06:26:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:26:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:26:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:26:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 06:26:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 06:26:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:26:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:26:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 06:26:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:26:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:26:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:26:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:26:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:26:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:26:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:26:59 [DEBUG   ] Finished updating board flags in 1.196 seconds
2018-11-27 06:27:14 [DEBUG   ] Got command BAM from MCS: ref #641128
2018-11-27 06:27:14 [INFO    ] Finished processing BAM request
2018-11-27 06:27:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:27:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641128   8 58449 23234029 A  ERROR'
2018-11-27 06:27:14 [DEBUG   ] Got command BAM from MCS: ref #641127
2018-11-27 06:27:14 [INFO    ] Finished processing BAM request
2018-11-27 06:27:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:27:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641127   8 58449 23234033 A  ERROR'
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:27:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:27:14 [DEBUG   ] spawn_process: at 2018-11-27 06:27:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:27:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:27:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:27:44 [DEBUG   ] Got command BAM from MCS: ref #641130
2018-11-27 06:27:44 [INFO    ] Finished processing BAM request
2018-11-27 06:27:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:27:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641130   8 58449 23264025 A  ERROR'
2018-11-27 06:27:44 [DEBUG   ] Got command BAM from MCS: ref #641129
2018-11-27 06:27:44 [INFO    ] Finished processing BAM request
2018-11-27 06:27:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:27:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641129   8 58449 23264029 A  ERROR'
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:27:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:27:45 [DEBUG   ] spawn_process: at 2018-11-27 06:27:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:27:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:27:58 [DEBUG   ] Got command RPT from MCS: ref #641131
2018-11-27 06:27:58 [DEBUG   ] summary = ERROR
2018-11-27 06:27:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641131  13 58449 23278941 A  ERRORERROR'
2018-11-27 06:27:59 [DEBUG   ] Got command RPT from MCS: ref #641132
2018-11-27 06:27:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:27:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641132  77 58449 23279145 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:27:59 [DEBUG   ] Got command RPT from MCS: ref #641133
2018-11-27 06:27:59 [DEBUG   ] summary = ERROR
2018-11-27 06:27:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641133  13 58449 23279353 A  ERRORERROR'
2018-11-27 06:27:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:27:59 [DEBUG   ] Got command RPT from MCS: ref #641134
2018-11-27 06:27:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:27:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641134  77 58449 23279545 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:27:59 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:27:59 [DEBUG   ] Got command RPT from MCS: ref #641135
2018-11-27 06:27:59 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:27:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641135  12 58449 23279749 A  ERRORB¡ff'
2018-11-27 06:27:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:27:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:27:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:27:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:27:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:27:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:27:59 [DEBUG   ] Finished updating board flags in 1.016 seconds
2018-11-27 06:28:14 [DEBUG   ] Got command BAM from MCS: ref #641140
2018-11-27 06:28:14 [INFO    ] Finished processing BAM request
2018-11-27 06:28:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:28:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641140   8 58449 23294033 A  ERROR'
2018-11-27 06:28:14 [DEBUG   ] Got command BAM from MCS: ref #641141
2018-11-27 06:28:14 [INFO    ] Finished processing BAM request
2018-11-27 06:28:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:28:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641141   8 58449 23294037 A  ERROR'
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:28:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:28:14 [DEBUG   ] spawn_process: at 2018-11-27 06:28:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:28:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:28:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:28:44 [DEBUG   ] Got command BAM from MCS: ref #641142
2018-11-27 06:28:44 [INFO    ] Finished processing BAM request
2018-11-27 06:28:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:28:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641142   8 58449 23324033 A  ERROR'
2018-11-27 06:28:44 [DEBUG   ] Got command BAM from MCS: ref #641143
2018-11-27 06:28:44 [INFO    ] Finished processing BAM request
2018-11-27 06:28:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:28:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641143   8 58449 23324041 A  ERROR'
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:28:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:28:44 [DEBUG   ] spawn_process: at 2018-11-27 06:28:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:28:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:28:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:28:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:28:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:28:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:28:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:28:59 [WARNING ] (13, 1, 'bot')
2018-11-27 06:28:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:28:59 [DEBUG   ] Finished updating board flags in 0.974 seconds
2018-11-27 06:29:14 [DEBUG   ] Got command BAM from MCS: ref #641145
2018-11-27 06:29:14 [INFO    ] Finished processing BAM request
2018-11-27 06:29:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:29:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641145   8 58449 23354029 A  ERROR'
2018-11-27 06:29:14 [DEBUG   ] Got command BAM from MCS: ref #641144
2018-11-27 06:29:14 [INFO    ] Finished processing BAM request
2018-11-27 06:29:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:29:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641144   8 58449 23354033 A  ERROR'
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:29:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:29:14 [DEBUG   ] spawn_process: at 2018-11-27 06:29:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:29:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:29:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:29:44 [DEBUG   ] Got command BAM from MCS: ref #641147
2018-11-27 06:29:44 [INFO    ] Finished processing BAM request
2018-11-27 06:29:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:29:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641147   8 58449 23384033 A  ERROR'
2018-11-27 06:29:44 [DEBUG   ] Got command BAM from MCS: ref #641146
2018-11-27 06:29:44 [INFO    ] Finished processing BAM request
2018-11-27 06:29:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:29:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641146   8 58449 23384041 A  ERROR'
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:29:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:29:45 [DEBUG   ] spawn_process: at 2018-11-27 06:29:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:29:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:29:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:29:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:30:00 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:30:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:30:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:30:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:30:00 [DEBUG   ] Finished updating board flags in 1.364 seconds
2018-11-27 06:30:14 [DEBUG   ] Got command BAM from MCS: ref #641148
2018-11-27 06:30:14 [INFO    ] Finished processing BAM request
2018-11-27 06:30:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:30:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641148   8 58449 23414029 A  ERROR'
2018-11-27 06:30:14 [DEBUG   ] Got command BAM from MCS: ref #641149
2018-11-27 06:30:14 [INFO    ] Finished processing BAM request
2018-11-27 06:30:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:30:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641149   8 58449 23414033 A  ERROR'
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:30:14 [DEBUG   ] spawn_process: at 2018-11-27 06:30:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:30:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:30:37 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 06:30:44 [DEBUG   ] Got command BAM from MCS: ref #641150
2018-11-27 06:30:44 [INFO    ] Finished processing BAM request
2018-11-27 06:30:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:30:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641150   8 58449 23444033 A  ERROR'
2018-11-27 06:30:44 [DEBUG   ] Got command BAM from MCS: ref #641151
2018-11-27 06:30:44 [INFO    ] Finished processing BAM request
2018-11-27 06:30:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:30:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641151   8 58449 23444037 A  ERROR'
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:30:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:30:44 [DEBUG   ] spawn_process: at 2018-11-27 06:30:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:30:48 [DEBUG   ] Got command RPT from MCS: ref #641152
2018-11-27 06:30:48 [DEBUG   ] summary = ERROR
2018-11-27 06:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641152  13 58449 23448937 A  ERRORERROR'
2018-11-27 06:30:48 [DEBUG   ] Got command RPT from MCS: ref #641153
2018-11-27 06:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641153  77 58449 23448938 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:30:57 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 06:30:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:31:00 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:31:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:31:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:31:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:31:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:31:00 [DEBUG   ] Finished updating board flags in 1.189 seconds
2018-11-27 06:31:14 [DEBUG   ] Got command BAM from MCS: ref #641174
2018-11-27 06:31:14 [INFO    ] Finished processing BAM request
2018-11-27 06:31:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:31:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641174   8 58449 23474029 A  ERROR'
2018-11-27 06:31:14 [DEBUG   ] Got command BAM from MCS: ref #641173
2018-11-27 06:31:14 [INFO    ] Finished processing BAM request
2018-11-27 06:31:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:31:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641173   8 58449 23474033 A  ERROR'
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:31:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:31:14 [DEBUG   ] spawn_process: at 2018-11-27 06:31:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:31:17 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 06:31:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:31:44 [DEBUG   ] Got command BAM from MCS: ref #641176
2018-11-27 06:31:44 [INFO    ] Finished processing BAM request
2018-11-27 06:31:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:31:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641176   8 58449 23504033 A  ERROR'
2018-11-27 06:31:44 [DEBUG   ] Got command BAM from MCS: ref #641175
2018-11-27 06:31:44 [INFO    ] Finished processing BAM request
2018-11-27 06:31:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:31:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641175   8 58449 23504037 A  ERROR'
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:31:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:31:44 [DEBUG   ] spawn_process: at 2018-11-27 06:31:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:31:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:31:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 06:31:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:32:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:32:00 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:32:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:32:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:32:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:32:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:32:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:32:00 [DEBUG   ] Finished updating board flags in 1.067 seconds
2018-11-27 06:32:14 [DEBUG   ] Got command BAM from MCS: ref #641177
2018-11-27 06:32:14 [INFO    ] Finished processing BAM request
2018-11-27 06:32:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:32:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641177   8 58449 23534033 A  ERROR'
2018-11-27 06:32:14 [DEBUG   ] Got command BAM from MCS: ref #641178
2018-11-27 06:32:14 [INFO    ] Finished processing BAM request
2018-11-27 06:32:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:32:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641178   8 58449 23534037 A  ERROR'
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:32:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:32:14 [DEBUG   ] spawn_process: at 2018-11-27 06:32:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:32:17 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:32:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:32:44 [DEBUG   ] Got command BAM from MCS: ref #641181
2018-11-27 06:32:44 [INFO    ] Finished processing BAM request
2018-11-27 06:32:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:32:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641181   8 58449 23564025 A  ERROR'
2018-11-27 06:32:44 [DEBUG   ] Got command BAM from MCS: ref #641182
2018-11-27 06:32:44 [INFO    ] Finished processing BAM request
2018-11-27 06:32:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:32:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641182   8 58449 23564029 A  ERROR'
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:32:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:32:44 [DEBUG   ] spawn_process: at 2018-11-27 06:32:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:32:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:32:59 [DEBUG   ] Got command RPT from MCS: ref #641183
2018-11-27 06:32:59 [DEBUG   ] summary = ERROR
2018-11-27 06:32:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641183  13 58449 23579033 A  ERRORERROR'
2018-11-27 06:32:59 [DEBUG   ] Got command RPT from MCS: ref #641184
2018-11-27 06:32:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:32:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641184  77 58449 23579237 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:32:59 [DEBUG   ] Got command RPT from MCS: ref #641185
2018-11-27 06:32:59 [DEBUG   ] summary = ERROR
2018-11-27 06:32:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641185  13 58449 23579429 A  ERRORERROR'
2018-11-27 06:32:59 [DEBUG   ] Got command RPT from MCS: ref #641186
2018-11-27 06:32:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:32:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641186  77 58449 23579633 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:32:59 [DEBUG   ] Got command RPT from MCS: ref #641188
2018-11-27 06:32:59 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:32:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641188  12 58449 23579837 A  ERRORB¢ff'
2018-11-27 06:33:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 06:33:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:33:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:33:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:33:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:33:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:33:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:33:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:33:01 [DEBUG   ] Finished updating board flags in 2.016 seconds
2018-11-27 06:33:14 [DEBUG   ] Got command BAM from MCS: ref #641193
2018-11-27 06:33:14 [INFO    ] Finished processing BAM request
2018-11-27 06:33:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:33:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641193   8 58449 23594033 A  ERROR'
2018-11-27 06:33:14 [DEBUG   ] Got command BAM from MCS: ref #641194
2018-11-27 06:33:14 [INFO    ] Finished processing BAM request
2018-11-27 06:33:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:33:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641194   8 58449 23594037 A  ERROR'
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:33:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:33:14 [DEBUG   ] spawn_process: at 2018-11-27 06:33:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:33:17 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:33:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:33:44 [DEBUG   ] Got command BAM from MCS: ref #641195
2018-11-27 06:33:44 [INFO    ] Finished processing BAM request
2018-11-27 06:33:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:33:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641195   8 58449 23624025 A  ERROR'
2018-11-27 06:33:44 [DEBUG   ] Got command BAM from MCS: ref #641196
2018-11-27 06:33:44 [INFO    ] Finished processing BAM request
2018-11-27 06:33:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:33:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641196   8 58449 23624029 A  ERROR'
2018-11-27 06:33:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:33:44 [DEBUG   ] spawn_process: at 2018-11-27 06:33:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:33:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:33:44 [DEBUG   ] spawn_process: at 2018-11-27 06:33:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:33:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:33:44 [DEBUG   ] spawn_process: at 2018-11-27 06:33:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:33:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:33:44 [DEBUG   ] spawn_process: at 2018-11-27 06:33:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:33:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:33:44 [DEBUG   ] spawn_process: at 2018-11-27 06:33:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:33:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:33:45 [DEBUG   ] spawn_process: at 2018-11-27 06:33:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:33:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:34:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:34:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 06:34:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2018-11-27 06:34:00 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:34:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:34:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:34:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:34:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:34:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:34:00 [DEBUG   ] Finished updating board flags in 1.359 seconds
2018-11-27 06:34:14 [DEBUG   ] Got command BAM from MCS: ref #641201
2018-11-27 06:34:14 [INFO    ] Finished processing BAM request
2018-11-27 06:34:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:34:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641201   8 58449 23654029 A  ERROR'
2018-11-27 06:34:14 [DEBUG   ] Got command BAM from MCS: ref #641202
2018-11-27 06:34:14 [INFO    ] Finished processing BAM request
2018-11-27 06:34:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:34:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641202   8 58449 23654033 A  ERROR'
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:34:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:34:14 [DEBUG   ] spawn_process: at 2018-11-27 06:34:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:34:17 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:34:37 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 06:34:46 [DEBUG   ] Got command STP from MCS: ref #641204
2018-11-27 06:34:46 [INFO    ] Finished processing BAM request
2018-11-27 06:34:46 [INFO    ] Shutdown the adder chain on beam #4
2018-11-27 06:34:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   641204   8 58449 23686033 A  ERROR'
2018-11-27 06:34:46 [DEBUG   ] Got command STP from MCS: ref #641203
2018-11-27 06:34:46 [INFO    ] Finished processing BAM request
2018-11-27 06:34:46 [INFO    ] Shutdown the adder chain on beam #3
2018-11-27 06:34:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   641203   8 58449 23686037 A  ERROR'
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:34:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:34:46 [DEBUG   ] spawn_process: at 2018-11-27 06:34:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:34:57 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 06:35:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:35:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:35:00 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:35:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:35:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:35:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:35:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:35:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:35:00 [DEBUG   ] Finished updating board flags in 1.219 seconds
2018-11-27 06:35:17 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 06:35:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 06:35:48 [DEBUG   ] Got command RPT from MCS: ref #641205
2018-11-27 06:35:48 [DEBUG   ] summary = ERROR
2018-11-27 06:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641205  13 58449 23748945 A  ERRORERROR'
2018-11-27 06:35:48 [DEBUG   ] Got command RPT from MCS: ref #641206
2018-11-27 06:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641206  77 58449 23748946 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:35:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 06:36:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:36:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:36:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:36:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:36:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:36:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:36:00 [DEBUG   ] Finished updating board flags in 1.220 seconds
2018-11-27 06:36:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 06:36:37 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 06:36:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:37:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:37:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:37:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:37:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:37:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:37:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:37:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:37:01 [DEBUG   ] Finished updating board flags in 1.740 seconds
2018-11-27 06:37:17 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 06:37:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:37:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:37:59 [DEBUG   ] Got command RPT from MCS: ref #641226
2018-11-27 06:37:59 [DEBUG   ] summary = ERROR
2018-11-27 06:37:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641226  13 58449 23879121 A  ERRORERROR'
2018-11-27 06:37:59 [DEBUG   ] Got command RPT from MCS: ref #641227
2018-11-27 06:37:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:37:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641227  77 58449 23879313 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:37:59 [DEBUG   ] Got command RPT from MCS: ref #641228
2018-11-27 06:37:59 [DEBUG   ] summary = ERROR
2018-11-27 06:37:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641228  13 58449 23879517 A  ERRORERROR'
2018-11-27 06:37:59 [DEBUG   ] Got command RPT from MCS: ref #641229
2018-11-27 06:37:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:37:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641229  77 58449 23879721 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:37:59 [DEBUG   ] Got command RPT from MCS: ref #641230
2018-11-27 06:37:59 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:37:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641230  12 58449 23879925 A  ERRORB›™š'
2018-11-27 06:38:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:38:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 06:38:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:38:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:38:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:38:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:38:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:38:00 [DEBUG   ] Finished updating board flags in 1.036 seconds
2018-11-27 06:38:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:38:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:38:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:39:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:39:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:39:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:39:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:39:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:39:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:39:00 [DEBUG   ] Finished updating board flags in 1.039 seconds
2018-11-27 06:39:13 [DEBUG   ] Got command DRX from MCS: ref #641235
2018-11-27 06:39:13 [INFO    ] sending drx command to lwa16
2018-11-27 06:39:13 [DEBUG   ] run_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 64500000.0 7 9 0']
2018-11-27 06:39:13 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 06:39:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   641235   8 58449 23953005 A  ERROR'
2018-11-27 06:39:13 [DEBUG   ] Got command DRX from MCS: ref #641236
2018-11-27 06:39:13 [INFO    ] sending drx command to lwa16
2018-11-27 06:39:13 [DEBUG   ] run_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 35100000.0 7 9 0']
2018-11-27 06:39:13 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 06:39:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   641236   8 58449 23953009 A  ERROR'
2018-11-27 06:39:13 [DEBUG   ] Got command DRX from MCS: ref #641237
2018-11-27 06:39:13 [INFO    ] sending drx command to lwa16
2018-11-27 06:39:13 [DEBUG   ] run_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 79200000.0 7 9 0']
2018-11-27 06:39:13 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 06:39:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   641237   8 58449 23953013 A  ERROR'
2018-11-27 06:39:13 [DEBUG   ] Got command DRX from MCS: ref #641238
2018-11-27 06:39:13 [INFO    ] sending drx command to lwa16
2018-11-27 06:39:13 [DEBUG   ] run_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 49800000.0 7 9 0']
2018-11-27 06:39:13 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 06:39:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   641238   8 58449 23953017 A  ERROR'
2018-11-27 06:39:13 [DEBUG   ] Got command BAM from MCS: ref #641239
2018-11-27 06:39:13 [INFO    ] Finished processing BAM request
2018-11-27 06:39:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:39:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641239   8 58449 23953021 A  ERROR'
2018-11-27 06:39:13 [DEBUG   ] Got command BAM from MCS: ref #641240
2018-11-27 06:39:13 [INFO    ] Finished processing BAM request
2018-11-27 06:39:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:39:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641240   8 58449 23953025 A  ERROR'
2018-11-27 06:39:13 [INFO    ] Finished starting DRX
2018-11-27 06:39:13 [INFO    ] Finished starting DRX
2018-11-27 06:39:13 [INFO    ] Finished starting DRX
2018-11-27 06:39:13 [INFO    ] Finished starting DRX
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:39:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:39:13 [DEBUG   ] spawn_process: at 2018-11-27 06:39:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:39:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:39:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:39:56 [DEBUG   ] Got command BAM from MCS: ref #641243
2018-11-27 06:39:56 [INFO    ] Finished processing BAM request
2018-11-27 06:39:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:39:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641243   8 58449 23996029 A  ERROR'
2018-11-27 06:39:56 [DEBUG   ] Got command BAM from MCS: ref #641244
2018-11-27 06:39:56 [INFO    ] Finished processing BAM request
2018-11-27 06:39:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:39:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641244   8 58449 23996033 A  ERROR'
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:39:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:39:56 [DEBUG   ] spawn_process: at 2018-11-27 06:39:56 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:39:57 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 06:40:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 06:40:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:40:00 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:40:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:40:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:40:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:40:00 [WARNING ] (13, 1, 'bot')
2018-11-27 06:40:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:40:00 [DEBUG   ] Finished updating board flags in 0.961 seconds
2018-11-27 06:40:17 [DEBUG   ] Finished updating ant stats in 0.590 seconds
2018-11-27 06:40:37 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:40:39 [DEBUG   ] Got command BAM from MCS: ref #641250
2018-11-27 06:40:39 [INFO    ] Finished processing BAM request
2018-11-27 06:40:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:40:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641250   8 58449 24039025 A  ERROR'
2018-11-27 06:40:39 [DEBUG   ] Got command BAM from MCS: ref #641249
2018-11-27 06:40:39 [INFO    ] Finished processing BAM request
2018-11-27 06:40:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:40:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641249   8 58449 24039029 A  ERROR'
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:40:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:40:39 [DEBUG   ] spawn_process: at 2018-11-27 06:40:39 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:40:48 [DEBUG   ] Got command RPT from MCS: ref #641251
2018-11-27 06:40:48 [DEBUG   ] summary = ERROR
2018-11-27 06:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641251  13 58449 24048941 A  ERRORERROR'
2018-11-27 06:40:48 [DEBUG   ] Got command RPT from MCS: ref #641252
2018-11-27 06:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641252  77 58449 24048942 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:40:57 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:41:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 06:41:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:41:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:41:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:41:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:41:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:41:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:41:01 [DEBUG   ] Finished updating board flags in 1.434 seconds
2018-11-27 06:41:17 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:41:22 [DEBUG   ] Got command BAM from MCS: ref #641268
2018-11-27 06:41:22 [INFO    ] Finished processing BAM request
2018-11-27 06:41:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:41:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641268   8 58449 24082033 A  ERROR'
2018-11-27 06:41:22 [DEBUG   ] Got command BAM from MCS: ref #641269
2018-11-27 06:41:22 [INFO    ] Finished processing BAM request
2018-11-27 06:41:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:41:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641269   8 58449 24082037 A  ERROR'
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:41:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:41:22 [DEBUG   ] spawn_process: at 2018-11-27 06:41:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:41:37 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:41:57 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 06:42:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:42:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:42:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 06:42:01 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:42:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:42:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:42:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:42:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:42:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:42:01 [DEBUG   ] Finished updating board flags in 1.338 seconds
2018-11-27 06:42:05 [DEBUG   ] Got command BAM from MCS: ref #641270
2018-11-27 06:42:05 [INFO    ] Finished processing BAM request
2018-11-27 06:42:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:42:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641270   8 58449 24125029 A  ERROR'
2018-11-27 06:42:05 [DEBUG   ] Got command BAM from MCS: ref #641271
2018-11-27 06:42:05 [INFO    ] Finished processing BAM request
2018-11-27 06:42:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:42:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641271   8 58449 24125033 A  ERROR'
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:42:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:42:05 [DEBUG   ] spawn_process: at 2018-11-27 06:42:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:42:17 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:42:37 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:42:47 [DEBUG   ] Got command BAM from MCS: ref #641275
2018-11-27 06:42:47 [INFO    ] Finished processing BAM request
2018-11-27 06:42:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:42:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641275   8 58449 24167025 A  ERROR'
2018-11-27 06:42:47 [DEBUG   ] Got command BAM from MCS: ref #641274
2018-11-27 06:42:47 [INFO    ] Finished processing BAM request
2018-11-27 06:42:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:42:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641274   8 58449 24167029 A  ERROR'
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:42:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:42:47 [DEBUG   ] spawn_process: at 2018-11-27 06:42:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:42:57 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:42:59 [DEBUG   ] Got command RPT from MCS: ref #641276
2018-11-27 06:42:59 [DEBUG   ] summary = ERROR
2018-11-27 06:42:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641276  13 58449 24179197 A  ERRORERROR'
2018-11-27 06:42:59 [DEBUG   ] Got command RPT from MCS: ref #641277
2018-11-27 06:42:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:42:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641277  77 58449 24179401 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:42:59 [DEBUG   ] Got command RPT from MCS: ref #641278
2018-11-27 06:42:59 [DEBUG   ] summary = ERROR
2018-11-27 06:42:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641278  13 58449 24179605 A  ERRORERROR'
2018-11-27 06:42:59 [DEBUG   ] Got command RPT from MCS: ref #641280
2018-11-27 06:42:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:42:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641280  77 58449 24179809 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:43:00 [DEBUG   ] Got command RPT from MCS: ref #641282
2018-11-27 06:43:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:43:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641282  12 58449 24180001 A  ERRORB ff'
2018-11-27 06:43:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:43:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:43:01 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:43:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:43:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:43:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:43:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:43:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:43:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:43:01 [DEBUG   ] Finished updating board flags in 1.203 seconds
2018-11-27 06:43:17 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:43:29 [DEBUG   ] Got command BAM from MCS: ref #641290
2018-11-27 06:43:29 [INFO    ] Finished processing BAM request
2018-11-27 06:43:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:43:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641290   8 58449 24209029 A  ERROR'
2018-11-27 06:43:29 [DEBUG   ] Got command BAM from MCS: ref #641291
2018-11-27 06:43:29 [INFO    ] Finished processing BAM request
2018-11-27 06:43:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:43:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641291   8 58449 24209033 A  ERROR'
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:43:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:43:29 [DEBUG   ] spawn_process: at 2018-11-27 06:43:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:43:37 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:43:57 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 06:44:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:44:01 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:44:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:44:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:44:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:44:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:44:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:44:01 [DEBUG   ] Finished updating board flags in 1.197 seconds
2018-11-27 06:44:11 [DEBUG   ] Got command BAM from MCS: ref #641293
2018-11-27 06:44:11 [INFO    ] Finished processing BAM request
2018-11-27 06:44:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:44:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641293   8 58449 24251025 A  ERROR'
2018-11-27 06:44:11 [DEBUG   ] Got command BAM from MCS: ref #641292
2018-11-27 06:44:11 [INFO    ] Finished processing BAM request
2018-11-27 06:44:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:44:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641292   8 58449 24251029 A  ERROR'
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:44:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:44:11 [DEBUG   ] spawn_process: at 2018-11-27 06:44:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:44:17 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:44:37 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:44:53 [DEBUG   ] Got command BAM from MCS: ref #641294
2018-11-27 06:44:53 [INFO    ] Finished processing BAM request
2018-11-27 06:44:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:44:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641294   8 58449 24293025 A  ERROR'
2018-11-27 06:44:53 [DEBUG   ] Got command BAM from MCS: ref #641295
2018-11-27 06:44:53 [INFO    ] Finished processing BAM request
2018-11-27 06:44:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:44:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641295   8 58449 24293029 A  ERROR'
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:44:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:44:53 [DEBUG   ] spawn_process: at 2018-11-27 06:44:53 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:44:57 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:45:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:45:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 06:45:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:45:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 06:45:01 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:45:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:45:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:45:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:45:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:45:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:45:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:45:01 [DEBUG   ] Finished updating board flags in 1.007 seconds
2018-11-27 06:45:17 [DEBUG   ] Finished updating ant stats in 0.399 seconds
2018-11-27 06:45:35 [DEBUG   ] Got command BAM from MCS: ref #641296
2018-11-27 06:45:35 [INFO    ] Finished processing BAM request
2018-11-27 06:45:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:45:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641296   8 58449 24335025 A  ERROR'
2018-11-27 06:45:35 [DEBUG   ] Got command BAM from MCS: ref #641297
2018-11-27 06:45:35 [INFO    ] Finished processing BAM request
2018-11-27 06:45:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:45:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641297   8 58449 24335029 A  ERROR'
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:45:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:45:35 [DEBUG   ] spawn_process: at 2018-11-27 06:45:35 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:45:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 06:45:48 [DEBUG   ] Got command RPT from MCS: ref #641298
2018-11-27 06:45:48 [DEBUG   ] summary = ERROR
2018-11-27 06:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641298  13 58449 24348949 A  ERRORERROR'
2018-11-27 06:45:48 [DEBUG   ] Got command RPT from MCS: ref #641299
2018-11-27 06:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641299  77 58449 24348950 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:45:58 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 06:46:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 06:46:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:46:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:46:01 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:46:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:46:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:46:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:46:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:46:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:46:01 [DEBUG   ] Finished updating board flags in 1.013 seconds
2018-11-27 06:46:17 [DEBUG   ] Got command BAM from MCS: ref #641320
2018-11-27 06:46:17 [INFO    ] Finished processing BAM request
2018-11-27 06:46:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:46:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641320   8 58449 24377029 A  ERROR'
2018-11-27 06:46:17 [DEBUG   ] Got command BAM from MCS: ref #641319
2018-11-27 06:46:17 [INFO    ] Finished processing BAM request
2018-11-27 06:46:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:46:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641319   8 58449 24377033 A  ERROR'
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:46:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:46:17 [DEBUG   ] spawn_process: at 2018-11-27 06:46:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:46:18 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 06:46:38 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 06:46:58 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 06:46:59 [DEBUG   ] Got command BAM from MCS: ref #641322
2018-11-27 06:46:59 [INFO    ] Finished processing BAM request
2018-11-27 06:46:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:46:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641322   8 58449 24419033 A  ERROR'
2018-11-27 06:46:59 [DEBUG   ] Got command BAM from MCS: ref #641321
2018-11-27 06:46:59 [INFO    ] Finished processing BAM request
2018-11-27 06:46:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:46:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641321   8 58449 24419037 A  ERROR'
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:46:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:46:59 [DEBUG   ] spawn_process: at 2018-11-27 06:46:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:47:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:47:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:47:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:47:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:47:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:47:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:47:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:47:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:47:01 [DEBUG   ] Finished updating board flags in 0.982 seconds
2018-11-27 06:47:18 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 06:47:38 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 06:47:41 [DEBUG   ] Got command BAM from MCS: ref #641324
2018-11-27 06:47:41 [INFO    ] Finished processing BAM request
2018-11-27 06:47:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:47:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641324   8 58449 24461033 A  ERROR'
2018-11-27 06:47:41 [DEBUG   ] Got command BAM from MCS: ref #641323
2018-11-27 06:47:41 [INFO    ] Finished processing BAM request
2018-11-27 06:47:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:47:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641323   8 58449 24461037 A  ERROR'
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:47:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:47:41 [DEBUG   ] spawn_process: at 2018-11-27 06:47:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:47:58 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:47:59 [DEBUG   ] Got command RPT from MCS: ref #641325
2018-11-27 06:47:59 [DEBUG   ] summary = ERROR
2018-11-27 06:47:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641325  13 58449 24479289 A  ERRORERROR'
2018-11-27 06:47:59 [DEBUG   ] Got command RPT from MCS: ref #641326
2018-11-27 06:47:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:47:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641326  77 58449 24479493 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:47:59 [DEBUG   ] Got command RPT from MCS: ref #641327
2018-11-27 06:47:59 [DEBUG   ] summary = ERROR
2018-11-27 06:47:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641327  13 58449 24479685 A  ERRORERROR'
2018-11-27 06:47:59 [DEBUG   ] Got command RPT from MCS: ref #641328
2018-11-27 06:47:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:47:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641328  77 58449 24479889 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:48:00 [DEBUG   ] Got command RPT from MCS: ref #641329
2018-11-27 06:48:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:48:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641329  12 58449 24480093 A  ERRORB¢ff'
2018-11-27 06:48:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:48:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:48:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:48:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:48:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:48:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:48:02 [DEBUG   ] Finished updating board flags in 1.812 seconds
2018-11-27 06:48:18 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 06:48:23 [DEBUG   ] Got command BAM from MCS: ref #641331
2018-11-27 06:48:23 [INFO    ] Finished processing BAM request
2018-11-27 06:48:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:48:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641331   8 58449 24503029 A  ERROR'
2018-11-27 06:48:23 [DEBUG   ] Got command BAM from MCS: ref #641330
2018-11-27 06:48:23 [INFO    ] Finished processing BAM request
2018-11-27 06:48:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:48:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641330   8 58449 24503033 A  ERROR'
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:48:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:48:23 [DEBUG   ] spawn_process: at 2018-11-27 06:48:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:48:38 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:48:58 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 06:49:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:49:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:49:01 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:49:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:49:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:49:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:49:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:49:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:49:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:49:01 [DEBUG   ] Finished updating board flags in 1.248 seconds
2018-11-27 06:49:05 [DEBUG   ] Got command BAM from MCS: ref #641333
2018-11-27 06:49:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:49:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641333   8 58449 24545025 A  ERROR'
2018-11-27 06:49:05 [DEBUG   ] Got command BAM from MCS: ref #641332
2018-11-27 06:49:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:49:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641332   8 58449 24545029 A  ERROR'
2018-11-27 06:49:05 [INFO    ] Finished processing BAM request
2018-11-27 06:49:05 [INFO    ] Finished processing BAM request
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:49:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:49:05 [DEBUG   ] spawn_process: at 2018-11-27 06:49:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:49:18 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:49:38 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:49:47 [DEBUG   ] Got command BAM from MCS: ref #641339
2018-11-27 06:49:47 [INFO    ] Finished processing BAM request
2018-11-27 06:49:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:49:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641339   8 58449 24587029 A  ERROR'
2018-11-27 06:49:47 [DEBUG   ] Got command BAM from MCS: ref #641338
2018-11-27 06:49:47 [INFO    ] Finished processing BAM request
2018-11-27 06:49:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:49:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641338   8 58449 24587033 A  ERROR'
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:49:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:49:47 [DEBUG   ] spawn_process: at 2018-11-27 06:49:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:49:58 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 06:50:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:50:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:50:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:50:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:50:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:50:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:50:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:50:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:50:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:50:01 [DEBUG   ] Finished updating board flags in 1.235 seconds
2018-11-27 06:50:18 [DEBUG   ] Finished updating ant stats in 0.386 seconds
2018-11-27 06:50:29 [DEBUG   ] Got command BAM from MCS: ref #641341
2018-11-27 06:50:29 [INFO    ] Finished processing BAM request
2018-11-27 06:50:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:50:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641341   8 58449 24629029 A  ERROR'
2018-11-27 06:50:29 [DEBUG   ] Got command BAM from MCS: ref #641340
2018-11-27 06:50:29 [INFO    ] Finished processing BAM request
2018-11-27 06:50:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:50:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641340   8 58449 24629033 A  ERROR'
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:50:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:50:29 [DEBUG   ] spawn_process: at 2018-11-27 06:50:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:50:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:50:48 [DEBUG   ] Got command RPT from MCS: ref #641342
2018-11-27 06:50:48 [DEBUG   ] summary = ERROR
2018-11-27 06:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641342  13 58449 24648954 A  ERRORERROR'
2018-11-27 06:50:48 [DEBUG   ] Got command RPT from MCS: ref #641343
2018-11-27 06:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641343  77 58449 24648954 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:50:59 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:51:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:51:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:51:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:51:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 06:51:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:51:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:51:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:51:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:51:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:51:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:51:01 [DEBUG   ] Finished updating board flags in 1.153 seconds
2018-11-27 06:51:11 [DEBUG   ] Got command BAM from MCS: ref #641360
2018-11-27 06:51:11 [INFO    ] Finished processing BAM request
2018-11-27 06:51:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:51:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641360   8 58449 24671033 A  ERROR'
2018-11-27 06:51:11 [DEBUG   ] Got command BAM from MCS: ref #641359
2018-11-27 06:51:11 [INFO    ] Finished processing BAM request
2018-11-27 06:51:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:51:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641359   8 58449 24671037 A  ERROR'
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:51:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:51:11 [DEBUG   ] spawn_process: at 2018-11-27 06:51:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:51:19 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:51:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:51:53 [DEBUG   ] Got command BAM from MCS: ref #641362
2018-11-27 06:51:53 [INFO    ] Finished processing BAM request
2018-11-27 06:51:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:51:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641362   8 58449 24713025 A  ERROR'
2018-11-27 06:51:53 [DEBUG   ] Got command BAM from MCS: ref #641361
2018-11-27 06:51:53 [INFO    ] Finished processing BAM request
2018-11-27 06:51:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:51:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641361   8 58449 24713029 A  ERROR'
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:51:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:51:53 [DEBUG   ] spawn_process: at 2018-11-27 06:51:53 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:51:59 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:52:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:52:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:52:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 06:52:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:52:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:52:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:52:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:52:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:52:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:52:01 [DEBUG   ] Finished updating board flags in 1.030 seconds
2018-11-27 06:52:19 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:52:35 [DEBUG   ] Got command BAM from MCS: ref #641368
2018-11-27 06:52:35 [INFO    ] Finished processing BAM request
2018-11-27 06:52:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:52:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641368   8 58449 24755033 A  ERROR'
2018-11-27 06:52:35 [DEBUG   ] Got command BAM from MCS: ref #641367
2018-11-27 06:52:35 [INFO    ] Finished processing BAM request
2018-11-27 06:52:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:52:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641367   8 58449 24755037 A  ERROR'
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:52:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:52:35 [DEBUG   ] spawn_process: at 2018-11-27 06:52:35 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:52:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:52:59 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 06:52:59 [DEBUG   ] Got command RPT from MCS: ref #641371
2018-11-27 06:52:59 [DEBUG   ] summary = ERROR
2018-11-27 06:52:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641371  13 58449 24779377 A  ERRORERROR'
2018-11-27 06:52:59 [DEBUG   ] Got command RPT from MCS: ref #641372
2018-11-27 06:52:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:52:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641372  77 58449 24779581 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:52:59 [DEBUG   ] Got command RPT from MCS: ref #641373
2018-11-27 06:52:59 [DEBUG   ] summary = ERROR
2018-11-27 06:52:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641373  13 58449 24779773 A  ERRORERROR'
2018-11-27 06:52:59 [DEBUG   ] Got command RPT from MCS: ref #641375
2018-11-27 06:52:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:52:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641375  77 58449 24779981 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:53:00 [DEBUG   ] Got command RPT from MCS: ref #641377
2018-11-27 06:53:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:53:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641377  12 58449 24780185 A  ERRORB¢ff'
2018-11-27 06:53:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:53:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:53:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:53:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 06:53:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:53:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:53:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:53:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:53:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:53:01 [DEBUG   ] Finished updating board flags in 1.019 seconds
2018-11-27 06:53:17 [DEBUG   ] Got command BAM from MCS: ref #641382
2018-11-27 06:53:17 [INFO    ] Finished processing BAM request
2018-11-27 06:53:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:53:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641382   8 58449 24797029 A  ERROR'
2018-11-27 06:53:17 [DEBUG   ] Got command BAM from MCS: ref #641381
2018-11-27 06:53:17 [INFO    ] Finished processing BAM request
2018-11-27 06:53:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:53:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641381   8 58449 24797033 A  ERROR'
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:53:17 [DEBUG   ] spawn_process: at 2018-11-27 06:53:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:53:19 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:53:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:53:59 [DEBUG   ] Got command BAM from MCS: ref #641383
2018-11-27 06:53:59 [INFO    ] Finished processing BAM request
2018-11-27 06:53:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:53:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641383   8 58449 24839033 A  ERROR'
2018-11-27 06:53:59 [DEBUG   ] Got command BAM from MCS: ref #641384
2018-11-27 06:53:59 [INFO    ] Finished processing BAM request
2018-11-27 06:53:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:53:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641384   8 58449 24839037 A  ERROR'
2018-11-27 06:53:59 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:53:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:53:59 [DEBUG   ] spawn_process: at 2018-11-27 06:53:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:54:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:54:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:54:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 06:54:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:54:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:54:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:54:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:54:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:54:01 [DEBUG   ] Finished updating board flags in 1.007 seconds
2018-11-27 06:54:19 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:54:39 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:54:41 [DEBUG   ] Got command BAM from MCS: ref #641385
2018-11-27 06:54:41 [INFO    ] Finished processing BAM request
2018-11-27 06:54:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:54:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641385   8 58449 24881029 A  ERROR'
2018-11-27 06:54:41 [DEBUG   ] Got command BAM from MCS: ref #641386
2018-11-27 06:54:41 [INFO    ] Finished processing BAM request
2018-11-27 06:54:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:54:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641386   8 58449 24881033 A  ERROR'
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:54:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:54:41 [DEBUG   ] spawn_process: at 2018-11-27 06:54:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:54:59 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:55:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 06:55:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:55:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:55:01 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:55:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:55:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:55:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:55:01 [WARNING ] (13, 1, 'bot')
2018-11-27 06:55:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:55:01 [DEBUG   ] Finished updating board flags in 0.972 seconds
2018-11-27 06:55:19 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 06:55:23 [DEBUG   ] Got command BAM from MCS: ref #641391
2018-11-27 06:55:23 [INFO    ] Finished processing BAM request
2018-11-27 06:55:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:55:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641391   8 58449 24923025 A  ERROR'
2018-11-27 06:55:23 [DEBUG   ] Got command BAM from MCS: ref #641392
2018-11-27 06:55:23 [INFO    ] Finished processing BAM request
2018-11-27 06:55:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:55:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641392   8 58449 24923029 A  ERROR'
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:55:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:55:23 [DEBUG   ] spawn_process: at 2018-11-27 06:55:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:55:39 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 06:55:48 [DEBUG   ] Got command RPT from MCS: ref #641393
2018-11-27 06:55:48 [DEBUG   ] summary = ERROR
2018-11-27 06:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641393  13 58449 24948941 A  ERRORERROR'
2018-11-27 06:55:48 [DEBUG   ] Got command RPT from MCS: ref #641394
2018-11-27 06:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641394  77 58449 24948942 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:55:59 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 06:56:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 06:56:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:56:02 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 06:56:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:56:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:56:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:56:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:56:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:56:02 [DEBUG   ] Finished updating board flags in 1.347 seconds
2018-11-27 06:56:05 [DEBUG   ] Got command BAM from MCS: ref #641410
2018-11-27 06:56:05 [INFO    ] Finished processing BAM request
2018-11-27 06:56:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:56:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641410   8 58449 24965025 A  ERROR'
2018-11-27 06:56:05 [DEBUG   ] Got command BAM from MCS: ref #641411
2018-11-27 06:56:05 [INFO    ] Finished processing BAM request
2018-11-27 06:56:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:56:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641411   8 58449 24965029 A  ERROR'
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 06:56:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 06:56:05 [DEBUG   ] spawn_process: at 2018-11-27 06:56:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 06:56:19 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 06:56:39 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 06:56:46 [DEBUG   ] Got command BAM from MCS: ref #641413
2018-11-27 06:56:46 [INFO    ] Finished processing BAM request
2018-11-27 06:56:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:56:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641413   8 58449 25006033 A  ERROR'
2018-11-27 06:56:46 [DEBUG   ] Got command BAM from MCS: ref #641412
2018-11-27 06:56:46 [INFO    ] Finished processing BAM request
2018-11-27 06:56:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:56:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641412   8 58449 25006037 A  ERROR'
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 06:56:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 06:56:46 [DEBUG   ] spawn_process: at 2018-11-27 06:56:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 06:56:59 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 06:57:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:57:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 06:57:02 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:57:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:57:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:57:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:57:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:57:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:57:02 [DEBUG   ] Finished updating board flags in 1.196 seconds
2018-11-27 06:57:19 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 06:57:27 [DEBUG   ] Got command BAM from MCS: ref #641414
2018-11-27 06:57:27 [INFO    ] Finished processing BAM request
2018-11-27 06:57:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:57:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641414   8 58449 25047033 A  ERROR'
2018-11-27 06:57:27 [DEBUG   ] Got command BAM from MCS: ref #641415
2018-11-27 06:57:27 [INFO    ] Finished processing BAM request
2018-11-27 06:57:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:57:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641415   8 58449 25047037 A  ERROR'
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:57:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:57:27 [DEBUG   ] spawn_process: at 2018-11-27 06:57:27 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:57:39 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 06:57:59 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 06:57:59 [DEBUG   ] Got command RPT from MCS: ref #641416
2018-11-27 06:57:59 [DEBUG   ] summary = ERROR
2018-11-27 06:57:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641416  13 58449 25079461 A  ERRORERROR'
2018-11-27 06:57:59 [DEBUG   ] Got command RPT from MCS: ref #641417
2018-11-27 06:57:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:57:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641417  77 58449 25079665 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:57:59 [DEBUG   ] Got command RPT from MCS: ref #641418
2018-11-27 06:57:59 [DEBUG   ] summary = ERROR
2018-11-27 06:57:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641418  13 58449 25079869 A  ERRORERROR'
2018-11-27 06:58:00 [DEBUG   ] Got command RPT from MCS: ref #641419
2018-11-27 06:58:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 06:58:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641419  77 58449 25080073 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 06:58:00 [DEBUG   ] Got command RPT from MCS: ref #641420
2018-11-27 06:58:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 06:58:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641420  12 58449 25080265 A  ERRORB¡ff'
2018-11-27 06:58:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:58:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:58:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2018-11-27 06:58:02 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:58:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:58:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:58:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:58:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:58:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:58:02 [DEBUG   ] Finished updating board flags in 0.989 seconds
2018-11-27 06:58:08 [DEBUG   ] Got command BAM from MCS: ref #641422
2018-11-27 06:58:08 [INFO    ] Finished processing BAM request
2018-11-27 06:58:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:58:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641422   8 58449 25088033 A  ERROR'
2018-11-27 06:58:08 [DEBUG   ] Got command BAM from MCS: ref #641421
2018-11-27 06:58:08 [INFO    ] Finished processing BAM request
2018-11-27 06:58:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:58:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641421   8 58449 25088037 A  ERROR'
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 06:58:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 06:58:08 [DEBUG   ] spawn_process: at 2018-11-27 06:58:08 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 06:58:19 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 06:58:39 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 06:58:49 [DEBUG   ] Got command BAM from MCS: ref #641428
2018-11-27 06:58:49 [INFO    ] Finished processing BAM request
2018-11-27 06:58:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:58:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641428   8 58449 25129025 A  ERROR'
2018-11-27 06:58:49 [DEBUG   ] Got command BAM from MCS: ref #641427
2018-11-27 06:58:49 [INFO    ] Finished processing BAM request
2018-11-27 06:58:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:58:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641427   8 58449 25129029 A  ERROR'
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 06:58:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 06:58:49 [DEBUG   ] spawn_process: at 2018-11-27 06:58:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 06:58:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:59:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 06:59:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 06:59:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 06:59:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2018-11-27 06:59:02 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 06:59:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:59:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:59:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:59:02 [WARNING ] (13, 1, 'bot')
2018-11-27 06:59:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 06:59:02 [DEBUG   ] Finished updating board flags in 1.265 seconds
2018-11-27 06:59:19 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 06:59:30 [DEBUG   ] Got command BAM from MCS: ref #641430
2018-11-27 06:59:30 [INFO    ] Finished processing BAM request
2018-11-27 06:59:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:59:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641430   8 58449 25170025 A  ERROR'
2018-11-27 06:59:30 [DEBUG   ] Got command BAM from MCS: ref #641429
2018-11-27 06:59:30 [INFO    ] Finished processing BAM request
2018-11-27 06:59:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 06:59:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641429   8 58449 25170029 A  ERROR'
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 06:59:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 06:59:30 [DEBUG   ] spawn_process: at 2018-11-27 06:59:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 06:59:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 06:59:59 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 07:00:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:00:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 07:00:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:00:03 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:00:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:00:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:00:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:00:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:00:03 [DEBUG   ] Finished updating board flags in 1.816 seconds
2018-11-27 07:00:11 [DEBUG   ] Got command BAM from MCS: ref #641432
2018-11-27 07:00:11 [INFO    ] Finished processing BAM request
2018-11-27 07:00:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:00:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641432   8 58449 25211033 A  ERROR'
2018-11-27 07:00:11 [DEBUG   ] Got command BAM from MCS: ref #641431
2018-11-27 07:00:11 [INFO    ] Finished processing BAM request
2018-11-27 07:00:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:00:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641431   8 58449 25211037 A  ERROR'
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:00:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:00:11 [DEBUG   ] spawn_process: at 2018-11-27 07:00:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:00:19 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 07:00:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 07:00:48 [DEBUG   ] Got command RPT from MCS: ref #641433
2018-11-27 07:00:48 [DEBUG   ] summary = ERROR
2018-11-27 07:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641433  13 58449 25248945 A  ERRORERROR'
2018-11-27 07:00:48 [DEBUG   ] Got command RPT from MCS: ref #641434
2018-11-27 07:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641434  77 58449 25248946 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:00:52 [DEBUG   ] Got command BAM from MCS: ref #641451
2018-11-27 07:00:52 [INFO    ] Finished processing BAM request
2018-11-27 07:00:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:00:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641451   8 58449 25252033 A  ERROR'
2018-11-27 07:00:52 [DEBUG   ] Got command BAM from MCS: ref #641450
2018-11-27 07:00:52 [INFO    ] Finished processing BAM request
2018-11-27 07:00:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:00:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641450   8 58449 25252037 A  ERROR'
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:00:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:00:52 [DEBUG   ] spawn_process: at 2018-11-27 07:00:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:00:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 07:01:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:01:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:01:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 07:01:02 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:01:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:01:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:01:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:01:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:01:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:01:02 [DEBUG   ] Finished updating board flags in 1.181 seconds
2018-11-27 07:01:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 07:01:33 [DEBUG   ] Got command BAM from MCS: ref #641457
2018-11-27 07:01:33 [INFO    ] Finished processing BAM request
2018-11-27 07:01:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:01:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641457   8 58449 25293029 A  ERROR'
2018-11-27 07:01:33 [DEBUG   ] Got command BAM from MCS: ref #641456
2018-11-27 07:01:33 [INFO    ] Finished processing BAM request
2018-11-27 07:01:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:01:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641456   8 58449 25293033 A  ERROR'
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:01:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:01:33 [DEBUG   ] spawn_process: at 2018-11-27 07:01:33 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:01:39 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 07:01:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 07:02:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:02:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 07:02:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:02:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:02:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 07:02:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:02:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:02:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:02:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:02:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:02:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:02:02 [DEBUG   ] Finished updating board flags in 1.034 seconds
2018-11-27 07:02:14 [DEBUG   ] Got command BAM from MCS: ref #641458
2018-11-27 07:02:14 [INFO    ] Finished processing BAM request
2018-11-27 07:02:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:02:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641458   8 58449 25334029 A  ERROR'
2018-11-27 07:02:14 [DEBUG   ] Got command BAM from MCS: ref #641459
2018-11-27 07:02:14 [INFO    ] Finished processing BAM request
2018-11-27 07:02:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:02:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641459   8 58449 25334033 A  ERROR'
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:02:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:02:14 [DEBUG   ] spawn_process: at 2018-11-27 07:02:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:02:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 07:02:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 07:02:55 [DEBUG   ] Got command BAM from MCS: ref #641463
2018-11-27 07:02:55 [INFO    ] Finished processing BAM request
2018-11-27 07:02:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:02:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641463   8 58449 25375025 A  ERROR'
2018-11-27 07:02:55 [DEBUG   ] Got command BAM from MCS: ref #641462
2018-11-27 07:02:55 [INFO    ] Finished processing BAM request
2018-11-27 07:02:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:02:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641462   8 58449 25375029 A  ERROR'
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:02:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:02:55 [DEBUG   ] spawn_process: at 2018-11-27 07:02:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:02:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 07:02:59 [DEBUG   ] Got command RPT from MCS: ref #641464
2018-11-27 07:02:59 [DEBUG   ] summary = ERROR
2018-11-27 07:02:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641464  13 58449 25379557 A  ERRORERROR'
2018-11-27 07:02:59 [DEBUG   ] Got command RPT from MCS: ref #641465
2018-11-27 07:02:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:02:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641465  77 58449 25379749 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:02:59 [DEBUG   ] Got command RPT from MCS: ref #641466
2018-11-27 07:02:59 [DEBUG   ] summary = ERROR
2018-11-27 07:02:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641466  13 58449 25379953 A  ERRORERROR'
2018-11-27 07:03:00 [DEBUG   ] Got command RPT from MCS: ref #641468
2018-11-27 07:03:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:03:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641468  77 58449 25380157 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:03:00 [DEBUG   ] Got command RPT from MCS: ref #641469
2018-11-27 07:03:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:03:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641469  12 58449 25380361 A  ERRORB¢ff'
2018-11-27 07:03:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 07:03:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 07:03:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:03:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:03:02 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:03:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:03:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:03:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:03:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:03:02 [WARNING ] (13, 1, 'bot')
2018-11-27 07:03:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:03:02 [DEBUG   ] Finished updating board flags in 0.997 seconds
2018-11-27 07:03:19 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 07:03:36 [DEBUG   ] Got command BAM from MCS: ref #641475
2018-11-27 07:03:36 [INFO    ] Finished processing BAM request
2018-11-27 07:03:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:03:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641475   8 58449 25416033 A  ERROR'
2018-11-27 07:03:36 [DEBUG   ] Got command BAM from MCS: ref #641474
2018-11-27 07:03:36 [INFO    ] Finished processing BAM request
2018-11-27 07:03:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:03:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641474   8 58449 25416037 A  ERROR'
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:03:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:03:36 [DEBUG   ] spawn_process: at 2018-11-27 07:03:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:03:39 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:03:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:04:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:04:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 07:04:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:04:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 07:04:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:04:03 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:04:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:04:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:04:03 [DEBUG   ] Finished updating board flags in 1.370 seconds
2018-11-27 07:04:17 [DEBUG   ] Got command BAM from MCS: ref #641480
2018-11-27 07:04:17 [INFO    ] Finished processing BAM request
2018-11-27 07:04:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:04:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641480   8 58449 25457025 A  ERROR'
2018-11-27 07:04:17 [DEBUG   ] Got command BAM from MCS: ref #641481
2018-11-27 07:04:17 [INFO    ] Finished processing BAM request
2018-11-27 07:04:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:04:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641481   8 58449 25457029 A  ERROR'
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:04:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:04:17 [DEBUG   ] spawn_process: at 2018-11-27 07:04:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:04:19 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:04:39 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:04:58 [DEBUG   ] Got command BAM from MCS: ref #641482
2018-11-27 07:04:58 [INFO    ] Finished processing BAM request
2018-11-27 07:04:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:04:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641482   8 58449 25498029 A  ERROR'
2018-11-27 07:04:58 [DEBUG   ] Got command BAM from MCS: ref #641483
2018-11-27 07:04:58 [INFO    ] Finished processing BAM request
2018-11-27 07:04:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:04:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641483   8 58449 25498037 A  ERROR'
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:04:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:04:58 [DEBUG   ] spawn_process: at 2018-11-27 07:04:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:04:59 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:05:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 07:05:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:05:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:05:03 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:05:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:05:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:05:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:05:03 [DEBUG   ] Finished updating board flags in 1.146 seconds
2018-11-27 07:05:19 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:05:39 [DEBUG   ] Got command BAM from MCS: ref #641485
2018-11-27 07:05:39 [INFO    ] Finished processing BAM request
2018-11-27 07:05:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:05:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641485   8 58449 25539025 A  ERROR'
2018-11-27 07:05:39 [DEBUG   ] Got command BAM from MCS: ref #641484
2018-11-27 07:05:39 [INFO    ] Finished processing BAM request
2018-11-27 07:05:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:05:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641484   8 58449 25539029 A  ERROR'
2018-11-27 07:05:39 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:05:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:05:39 [DEBUG   ] spawn_process: at 2018-11-27 07:05:39 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:05:48 [DEBUG   ] Got command RPT from MCS: ref #641486
2018-11-27 07:05:48 [DEBUG   ] summary = ERROR
2018-11-27 07:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641486  13 58449 25548953 A  ERRORERROR'
2018-11-27 07:05:48 [DEBUG   ] Got command RPT from MCS: ref #641487
2018-11-27 07:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641487  77 58449 25548954 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:05:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:06:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:06:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 07:06:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2018-11-27 07:06:03 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:06:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:06:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:06:03 [DEBUG   ] Finished updating board flags in 1.015 seconds
2018-11-27 07:06:19 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:06:20 [DEBUG   ] Got command BAM from MCS: ref #641503
2018-11-27 07:06:20 [INFO    ] Finished processing BAM request
2018-11-27 07:06:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:06:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641503   8 58449 25580025 A  ERROR'
2018-11-27 07:06:20 [DEBUG   ] Got command BAM from MCS: ref #641504
2018-11-27 07:06:20 [INFO    ] Finished processing BAM request
2018-11-27 07:06:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:06:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641504   8 58449 25580029 A  ERROR'
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:06:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:06:20 [DEBUG   ] spawn_process: at 2018-11-27 07:06:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:06:39 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:06:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:07:01 [DEBUG   ] Got command BAM from MCS: ref #641505
2018-11-27 07:07:01 [INFO    ] Finished processing BAM request
2018-11-27 07:07:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:07:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641505   8 58449 25621029 A  ERROR'
2018-11-27 07:07:01 [DEBUG   ] Got command BAM from MCS: ref #641506
2018-11-27 07:07:01 [INFO    ] Finished processing BAM request
2018-11-27 07:07:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:07:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641506   8 58449 25621033 A  ERROR'
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:07:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:07:01 [DEBUG   ] spawn_process: at 2018-11-27 07:07:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:07:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:07:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:07:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2018-11-27 07:07:03 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:07:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:07:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:07:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:07:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:07:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:07:03 [DEBUG   ] Finished updating board flags in 1.008 seconds
2018-11-27 07:07:19 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:07:39 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:07:42 [DEBUG   ] Got command BAM from MCS: ref #641512
2018-11-27 07:07:42 [INFO    ] Finished processing BAM request
2018-11-27 07:07:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:07:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641512   8 58449 25662029 A  ERROR'
2018-11-27 07:07:42 [DEBUG   ] Got command BAM from MCS: ref #641511
2018-11-27 07:07:42 [INFO    ] Finished processing BAM request
2018-11-27 07:07:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:07:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641511   8 58449 25662033 A  ERROR'
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:07:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:07:42 [DEBUG   ] spawn_process: at 2018-11-27 07:07:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:07:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:07:59 [DEBUG   ] Got command RPT from MCS: ref #641513
2018-11-27 07:07:59 [DEBUG   ] summary = ERROR
2018-11-27 07:07:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641513  13 58449 25679641 A  ERRORERROR'
2018-11-27 07:07:59 [DEBUG   ] Got command RPT from MCS: ref #641514
2018-11-27 07:07:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:07:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641514  77 58449 25679833 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:08:00 [DEBUG   ] Got command RPT from MCS: ref #641515
2018-11-27 07:08:00 [DEBUG   ] summary = ERROR
2018-11-27 07:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641515  13 58449 25680037 A  ERRORERROR'
2018-11-27 07:08:00 [DEBUG   ] Got command RPT from MCS: ref #641516
2018-11-27 07:08:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641516  77 58449 25680241 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:08:00 [DEBUG   ] Got command RPT from MCS: ref #641517
2018-11-27 07:08:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641517  12 58449 25680445 A  ERRORB¢ff'
2018-11-27 07:08:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 07:08:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:08:03 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:08:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:08:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:08:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:08:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:08:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:08:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:08:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:08:03 [DEBUG   ] Finished updating board flags in 1.004 seconds
2018-11-27 07:08:19 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:08:23 [DEBUG   ] Got command BAM from MCS: ref #641518
2018-11-27 07:08:23 [INFO    ] Finished processing BAM request
2018-11-27 07:08:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:08:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641518   8 58449 25703033 A  ERROR'
2018-11-27 07:08:23 [DEBUG   ] Got command BAM from MCS: ref #641519
2018-11-27 07:08:23 [INFO    ] Finished processing BAM request
2018-11-27 07:08:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:08:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641519   8 58449 25703037 A  ERROR'
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:08:23 [DEBUG   ] spawn_process: at 2018-11-27 07:08:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:08:39 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:08:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:09:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa10, chip 2
2018-11-27 07:09:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:09:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:09:03 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:09:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:09:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:09:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:09:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:09:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:09:03 [DEBUG   ] Finished updating board flags in 1.015 seconds
2018-11-27 07:09:04 [DEBUG   ] Got command BAM from MCS: ref #641520
2018-11-27 07:09:04 [INFO    ] Finished processing BAM request
2018-11-27 07:09:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:09:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641520   8 58449 25744025 A  ERROR'
2018-11-27 07:09:04 [DEBUG   ] Got command BAM from MCS: ref #641521
2018-11-27 07:09:04 [INFO    ] Finished processing BAM request
2018-11-27 07:09:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:09:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641521   8 58449 25744029 A  ERROR'
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:09:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:09:04 [DEBUG   ] spawn_process: at 2018-11-27 07:09:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:09:19 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:09:39 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:09:45 [DEBUG   ] Got command BAM from MCS: ref #641523
2018-11-27 07:09:45 [INFO    ] Finished processing BAM request
2018-11-27 07:09:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:09:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641523   8 58449 25785029 A  ERROR'
2018-11-27 07:09:45 [DEBUG   ] Got command BAM from MCS: ref #641522
2018-11-27 07:09:45 [INFO    ] Finished processing BAM request
2018-11-27 07:09:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:09:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641522   8 58449 25785033 A  ERROR'
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:09:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:09:45 [DEBUG   ] spawn_process: at 2018-11-27 07:09:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:09:59 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:10:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 07:10:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 07:10:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:10:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:10:03 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:10:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:10:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:10:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:10:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:10:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:10:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:10:03 [DEBUG   ] Finished updating board flags in 1.513 seconds
2018-11-27 07:10:19 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:10:26 [DEBUG   ] Got command BAM from MCS: ref #641529
2018-11-27 07:10:26 [INFO    ] Finished processing BAM request
2018-11-27 07:10:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:10:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641529   8 58449 25826029 A  ERROR'
2018-11-27 07:10:26 [DEBUG   ] Got command BAM from MCS: ref #641528
2018-11-27 07:10:26 [INFO    ] Finished processing BAM request
2018-11-27 07:10:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:10:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641528   8 58449 25826033 A  ERROR'
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:10:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:10:26 [DEBUG   ] spawn_process: at 2018-11-27 07:10:26 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:10:39 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:10:48 [DEBUG   ] Got command RPT from MCS: ref #641530
2018-11-27 07:10:48 [DEBUG   ] summary = ERROR
2018-11-27 07:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641530  13 58449 25848949 A  ERRORERROR'
2018-11-27 07:10:48 [DEBUG   ] Got command RPT from MCS: ref #641531
2018-11-27 07:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641531  77 58449 25848950 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:10:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:11:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:11:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:11:04 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:11:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:11:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:11:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:11:04 [DEBUG   ] Finished updating board flags in 1.933 seconds
2018-11-27 07:11:07 [DEBUG   ] Got command BAM from MCS: ref #641547
2018-11-27 07:11:07 [INFO    ] Finished processing BAM request
2018-11-27 07:11:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:11:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641547   8 58449 25867033 A  ERROR'
2018-11-27 07:11:07 [DEBUG   ] Got command BAM from MCS: ref #641548
2018-11-27 07:11:07 [INFO    ] Finished processing BAM request
2018-11-27 07:11:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:11:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641548   8 58449 25867037 A  ERROR'
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:11:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:11:07 [DEBUG   ] spawn_process: at 2018-11-27 07:11:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:11:19 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:11:39 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:11:48 [DEBUG   ] Got command BAM from MCS: ref #641549
2018-11-27 07:11:48 [INFO    ] Finished processing BAM request
2018-11-27 07:11:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:11:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641549   8 58449 25908033 A  ERROR'
2018-11-27 07:11:48 [DEBUG   ] Got command BAM from MCS: ref #641550
2018-11-27 07:11:48 [INFO    ] Finished processing BAM request
2018-11-27 07:11:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:11:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641550   8 58449 25908037 A  ERROR'
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:11:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:11:48 [DEBUG   ] spawn_process: at 2018-11-27 07:11:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:11:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:12:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:12:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:12:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:12:04 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:12:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:12:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:12:04 [DEBUG   ] Finished updating board flags in 1.732 seconds
2018-11-27 07:12:19 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:12:29 [DEBUG   ] Got command BAM from MCS: ref #641551
2018-11-27 07:12:29 [INFO    ] Finished processing BAM request
2018-11-27 07:12:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:12:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641551   8 58449 25949033 A  ERROR'
2018-11-27 07:12:29 [DEBUG   ] Got command BAM from MCS: ref #641552
2018-11-27 07:12:29 [INFO    ] Finished processing BAM request
2018-11-27 07:12:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:12:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641552   8 58449 25949037 A  ERROR'
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:12:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:12:29 [DEBUG   ] spawn_process: at 2018-11-27 07:12:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:12:39 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:12:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:12:59 [DEBUG   ] Got command RPT from MCS: ref #641555
2018-11-27 07:12:59 [DEBUG   ] summary = ERROR
2018-11-27 07:12:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641555  13 58449 25979717 A  ERRORERROR'
2018-11-27 07:12:59 [DEBUG   ] Got command RPT from MCS: ref #641556
2018-11-27 07:12:59 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:12:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641556  77 58449 25979921 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:13:00 [DEBUG   ] Got command RPT from MCS: ref #641557
2018-11-27 07:13:00 [DEBUG   ] summary = ERROR
2018-11-27 07:13:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641557  13 58449 25980125 A  ERRORERROR'
2018-11-27 07:13:00 [DEBUG   ] Got command RPT from MCS: ref #641558
2018-11-27 07:13:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:13:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641558  77 58449 25980330 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:13:00 [DEBUG   ] Got command RPT from MCS: ref #641560
2018-11-27 07:13:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:13:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641560  12 58449 25980521 A  ERRORB¢ff'
2018-11-27 07:13:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:13:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:13:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:13:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 07:13:04 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:13:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:13:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:13:04 [DEBUG   ] Finished updating board flags in 1.622 seconds
2018-11-27 07:13:10 [DEBUG   ] Got command BAM from MCS: ref #641565
2018-11-27 07:13:10 [INFO    ] Finished processing BAM request
2018-11-27 07:13:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:13:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641565   8 58449 25990029 A  ERROR'
2018-11-27 07:13:10 [DEBUG   ] Got command BAM from MCS: ref #641566
2018-11-27 07:13:10 [INFO    ] Finished processing BAM request
2018-11-27 07:13:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:13:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641566   8 58449 25990033 A  ERROR'
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:13:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:13:10 [DEBUG   ] spawn_process: at 2018-11-27 07:13:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:13:19 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:13:39 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:13:51 [DEBUG   ] Got command BAM from MCS: ref #641572
2018-11-27 07:13:51 [INFO    ] Finished processing BAM request
2018-11-27 07:13:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:13:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641572   8 58449 26031025 A  ERROR'
2018-11-27 07:13:51 [DEBUG   ] Got command BAM from MCS: ref #641571
2018-11-27 07:13:51 [INFO    ] Finished processing BAM request
2018-11-27 07:13:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:13:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641571   8 58449 26031029 A  ERROR'
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:13:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:13:51 [DEBUG   ] spawn_process: at 2018-11-27 07:13:51 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:13:59 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:14:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:14:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:14:03 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:14:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:14:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:14:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:14:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:14:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:14:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:14:03 [DEBUG   ] Finished updating board flags in 1.039 seconds
2018-11-27 07:14:19 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:14:32 [DEBUG   ] Got command BAM from MCS: ref #641573
2018-11-27 07:14:32 [INFO    ] Finished processing BAM request
2018-11-27 07:14:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:14:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641573   8 58449 26072029 A  ERROR'
2018-11-27 07:14:32 [DEBUG   ] Got command BAM from MCS: ref #641574
2018-11-27 07:14:32 [INFO    ] Finished processing BAM request
2018-11-27 07:14:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:14:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641574   8 58449 26072033 A  ERROR'
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:14:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:14:32 [DEBUG   ] spawn_process: at 2018-11-27 07:14:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:14:39 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:14:59 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:15:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:15:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 07:15:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:15:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:15:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 07:15:03 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:15:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:15:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:15:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:15:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:15:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:15:03 [DEBUG   ] Finished updating board flags in 1.023 seconds
2018-11-27 07:15:13 [DEBUG   ] Got command BAM from MCS: ref #641575
2018-11-27 07:15:13 [INFO    ] Finished processing BAM request
2018-11-27 07:15:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:15:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641575   8 58449 26113033 A  ERROR'
2018-11-27 07:15:13 [DEBUG   ] Got command BAM from MCS: ref #641576
2018-11-27 07:15:13 [INFO    ] Finished processing BAM request
2018-11-27 07:15:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:15:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641576   8 58449 26113037 A  ERROR'
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:15:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:15:13 [DEBUG   ] spawn_process: at 2018-11-27 07:15:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:15:19 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 07:15:40 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 07:15:48 [DEBUG   ] Got command RPT from MCS: ref #641577
2018-11-27 07:15:48 [DEBUG   ] summary = ERROR
2018-11-27 07:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641577  13 58449 26148957 A  ERRORERROR'
2018-11-27 07:15:48 [DEBUG   ] Got command RPT from MCS: ref #641578
2018-11-27 07:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641578  77 58449 26148958 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:15:54 [DEBUG   ] Got command BAM from MCS: ref #641595
2018-11-27 07:15:54 [INFO    ] Finished processing BAM request
2018-11-27 07:15:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:15:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641595   8 58449 26154033 A  ERROR'
2018-11-27 07:15:54 [DEBUG   ] Got command BAM from MCS: ref #641594
2018-11-27 07:15:54 [INFO    ] Finished processing BAM request
2018-11-27 07:15:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:15:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641594   8 58449 26154037 A  ERROR'
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:15:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:15:54 [DEBUG   ] spawn_process: at 2018-11-27 07:15:54 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:16:00 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 07:16:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:16:03 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:16:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:16:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:16:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:16:03 [WARNING ] (13, 1, 'bot')
2018-11-27 07:16:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:16:03 [DEBUG   ] Finished updating board flags in 0.961 seconds
2018-11-27 07:16:20 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 07:16:35 [DEBUG   ] Got command BAM from MCS: ref #641600
2018-11-27 07:16:35 [INFO    ] Finished processing BAM request
2018-11-27 07:16:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:16:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641600   8 58449 26195034 A  ERROR'
2018-11-27 07:16:35 [DEBUG   ] Got command BAM from MCS: ref #641601
2018-11-27 07:16:35 [INFO    ] Finished processing BAM request
2018-11-27 07:16:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:16:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641601   8 58449 26195038 A  ERROR'
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:16:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:16:35 [DEBUG   ] spawn_process: at 2018-11-27 07:16:35 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:16:40 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:17:00 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:17:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:17:04 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:17:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:17:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:17:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:17:04 [DEBUG   ] Finished updating board flags in 1.359 seconds
2018-11-27 07:17:16 [DEBUG   ] Got command BAM from MCS: ref #641603
2018-11-27 07:17:16 [INFO    ] Finished processing BAM request
2018-11-27 07:17:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:17:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641603   8 58449 26236033 A  ERROR'
2018-11-27 07:17:16 [DEBUG   ] Got command BAM from MCS: ref #641602
2018-11-27 07:17:16 [INFO    ] Finished processing BAM request
2018-11-27 07:17:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:17:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641602   8 58449 26236037 A  ERROR'
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:17:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:17:16 [DEBUG   ] spawn_process: at 2018-11-27 07:17:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:17:20 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:17:40 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:17:57 [DEBUG   ] Got command BAM from MCS: ref #641604
2018-11-27 07:17:57 [INFO    ] Finished processing BAM request
2018-11-27 07:17:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:17:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641604   8 58449 26277029 A  ERROR'
2018-11-27 07:17:57 [DEBUG   ] Got command BAM from MCS: ref #641605
2018-11-27 07:17:57 [INFO    ] Finished processing BAM request
2018-11-27 07:17:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:17:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641605   8 58449 26277033 A  ERROR'
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:17:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:17:57 [DEBUG   ] spawn_process: at 2018-11-27 07:17:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:17:59 [DEBUG   ] Got command RPT from MCS: ref #641606
2018-11-27 07:17:59 [DEBUG   ] summary = ERROR
2018-11-27 07:17:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641606  13 58449 26279813 A  ERRORERROR'
2018-11-27 07:18:00 [DEBUG   ] Got command RPT from MCS: ref #641607
2018-11-27 07:18:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:18:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641607  77 58449 26280017 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:18:00 [DEBUG   ] Got command RPT from MCS: ref #641608
2018-11-27 07:18:00 [DEBUG   ] summary = ERROR
2018-11-27 07:18:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641608  13 58449 26280209 A  ERRORERROR'
2018-11-27 07:18:00 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:18:00 [DEBUG   ] Got command RPT from MCS: ref #641609
2018-11-27 07:18:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:18:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641609  77 58449 26280413 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:18:00 [DEBUG   ] Got command RPT from MCS: ref #641610
2018-11-27 07:18:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:18:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641610  12 58449 26280617 A  ERRORB¢ff'
2018-11-27 07:18:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 07:18:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:18:04 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:18:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:18:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:18:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:18:04 [DEBUG   ] Finished updating board flags in 1.270 seconds
2018-11-27 07:18:20 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:18:37 [DEBUG   ] Got command BAM from MCS: ref #641612
2018-11-27 07:18:37 [INFO    ] Finished processing BAM request
2018-11-27 07:18:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:18:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641612   8 58449 26317029 A  ERROR'
2018-11-27 07:18:37 [DEBUG   ] Got command BAM from MCS: ref #641611
2018-11-27 07:18:37 [INFO    ] Finished processing BAM request
2018-11-27 07:18:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:18:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641611   8 58449 26317033 A  ERROR'
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:18:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:18:37 [DEBUG   ] spawn_process: at 2018-11-27 07:18:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:18:40 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:19:00 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:19:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:19:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 07:19:04 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:19:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:19:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:19:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:19:04 [DEBUG   ] Finished updating board flags in 1.150 seconds
2018-11-27 07:19:17 [DEBUG   ] Got command BAM from MCS: ref #641618
2018-11-27 07:19:17 [INFO    ] Finished processing BAM request
2018-11-27 07:19:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:19:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641618   8 58449 26357029 A  ERROR'
2018-11-27 07:19:17 [DEBUG   ] Got command BAM from MCS: ref #641617
2018-11-27 07:19:17 [INFO    ] Finished processing BAM request
2018-11-27 07:19:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:19:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641617   8 58449 26357033 A  ERROR'
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:19:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:19:17 [DEBUG   ] spawn_process: at 2018-11-27 07:19:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:19:20 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 07:19:40 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:19:57 [DEBUG   ] Got command BAM from MCS: ref #641619
2018-11-27 07:19:57 [INFO    ] Finished processing BAM request
2018-11-27 07:19:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:19:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641619   8 58449 26397025 A  ERROR'
2018-11-27 07:19:57 [DEBUG   ] Got command BAM from MCS: ref #641620
2018-11-27 07:19:57 [INFO    ] Finished processing BAM request
2018-11-27 07:19:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:19:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641620   8 58449 26397029 A  ERROR'
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:19:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:19:57 [DEBUG   ] spawn_process: at 2018-11-27 07:19:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:20:00 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 07:20:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:20:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 07:20:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 07:20:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:20:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 07:20:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:20:04 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:20:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:20:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:20:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:20:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:20:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:20:04 [DEBUG   ] Finished updating board flags in 1.011 seconds
2018-11-27 07:20:20 [DEBUG   ] Finished updating ant stats in 0.591 seconds
2018-11-27 07:20:37 [DEBUG   ] Got command BAM from MCS: ref #641621
2018-11-27 07:20:37 [INFO    ] Finished processing BAM request
2018-11-27 07:20:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:20:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641621   8 58449 26437029 A  ERROR'
2018-11-27 07:20:37 [DEBUG   ] Got command BAM from MCS: ref #641622
2018-11-27 07:20:37 [INFO    ] Finished processing BAM request
2018-11-27 07:20:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:20:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641622   8 58449 26437033 A  ERROR'
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:20:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:20:37 [DEBUG   ] spawn_process: at 2018-11-27 07:20:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:20:40 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:20:48 [DEBUG   ] Got command RPT from MCS: ref #641623
2018-11-27 07:20:48 [DEBUG   ] summary = ERROR
2018-11-27 07:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641623  13 58449 26448957 A  ERRORERROR'
2018-11-27 07:20:48 [DEBUG   ] Got command RPT from MCS: ref #641624
2018-11-27 07:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641624  77 58449 26448958 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:21:00 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:21:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:21:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 07:21:04 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:21:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:21:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:21:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:21:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:21:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:21:04 [DEBUG   ] Finished updating board flags in 1.455 seconds
2018-11-27 07:21:17 [DEBUG   ] Got command BAM from MCS: ref #641640
2018-11-27 07:21:17 [INFO    ] Finished processing BAM request
2018-11-27 07:21:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:21:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641640   8 58449 26477025 A  ERROR'
2018-11-27 07:21:17 [DEBUG   ] Got command BAM from MCS: ref #641641
2018-11-27 07:21:17 [INFO    ] Finished processing BAM request
2018-11-27 07:21:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:21:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641641   8 58449 26477029 A  ERROR'
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:21:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:21:17 [DEBUG   ] spawn_process: at 2018-11-27 07:21:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:21:20 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 07:21:41 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 07:21:57 [DEBUG   ] Got command BAM from MCS: ref #641642
2018-11-27 07:21:57 [INFO    ] Finished processing BAM request
2018-11-27 07:21:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:21:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641642   8 58449 26517029 A  ERROR'
2018-11-27 07:21:57 [DEBUG   ] Got command BAM from MCS: ref #641643
2018-11-27 07:21:57 [INFO    ] Finished processing BAM request
2018-11-27 07:21:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:21:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641643   8 58449 26517033 A  ERROR'
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:21:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:21:57 [DEBUG   ] spawn_process: at 2018-11-27 07:21:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:22:01 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 07:22:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:22:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:22:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:22:04 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:22:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:22:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:22:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:22:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:22:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:22:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:22:04 [DEBUG   ] Finished updating board flags in 1.405 seconds
2018-11-27 07:22:21 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 07:22:37 [DEBUG   ] Got command BAM from MCS: ref #641648
2018-11-27 07:22:37 [INFO    ] Finished processing BAM request
2018-11-27 07:22:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:22:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641648   8 58449 26557033 A  ERROR'
2018-11-27 07:22:37 [DEBUG   ] Got command BAM from MCS: ref #641649
2018-11-27 07:22:37 [INFO    ] Finished processing BAM request
2018-11-27 07:22:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:22:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641649   8 58449 26557037 A  ERROR'
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:22:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:22:37 [DEBUG   ] spawn_process: at 2018-11-27 07:22:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:22:41 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:22:59 [DEBUG   ] Got command RPT from MCS: ref #641652
2018-11-27 07:22:59 [DEBUG   ] summary = ERROR
2018-11-27 07:22:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641652  13 58449 26579893 A  ERRORERROR'
2018-11-27 07:23:00 [DEBUG   ] Got command RPT from MCS: ref #641653
2018-11-27 07:23:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:23:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641653  77 58449 26580101 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:23:00 [DEBUG   ] Got command RPT from MCS: ref #641654
2018-11-27 07:23:00 [DEBUG   ] summary = ERROR
2018-11-27 07:23:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641654  13 58449 26580293 A  ERRORERROR'
2018-11-27 07:23:00 [DEBUG   ] Got command RPT from MCS: ref #641656
2018-11-27 07:23:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:23:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641656  77 58449 26580497 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:23:00 [DEBUG   ] Got command RPT from MCS: ref #641657
2018-11-27 07:23:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:23:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641657  12 58449 26580701 A  ERRORB¢ff'
2018-11-27 07:23:01 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:23:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:23:04 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:23:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:23:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:23:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:23:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:23:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:23:04 [DEBUG   ] Finished updating board flags in 1.369 seconds
2018-11-27 07:23:17 [DEBUG   ] Got command BAM from MCS: ref #641662
2018-11-27 07:23:17 [INFO    ] Finished processing BAM request
2018-11-27 07:23:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:23:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641662   8 58449 26597025 A  ERROR'
2018-11-27 07:23:17 [DEBUG   ] Got command BAM from MCS: ref #641663
2018-11-27 07:23:17 [INFO    ] Finished processing BAM request
2018-11-27 07:23:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:23:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641663   8 58449 26597029 A  ERROR'
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:23:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:23:17 [DEBUG   ] spawn_process: at 2018-11-27 07:23:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:23:21 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:23:41 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:23:57 [DEBUG   ] Got command BAM from MCS: ref #641665
2018-11-27 07:23:57 [INFO    ] Finished processing BAM request
2018-11-27 07:23:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:23:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641665   8 58449 26637025 A  ERROR'
2018-11-27 07:23:57 [DEBUG   ] Got command BAM from MCS: ref #641664
2018-11-27 07:23:57 [INFO    ] Finished processing BAM request
2018-11-27 07:23:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:23:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641664   8 58449 26637029 A  ERROR'
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:23:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:23:57 [DEBUG   ] spawn_process: at 2018-11-27 07:23:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:24:01 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 07:24:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:24:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:24:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:24:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:24:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:24:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:24:05 [DEBUG   ] Finished updating board flags in 1.759 seconds
2018-11-27 07:24:21 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:24:37 [DEBUG   ] Got command BAM from MCS: ref #641666
2018-11-27 07:24:37 [INFO    ] Finished processing BAM request
2018-11-27 07:24:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:24:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641666   8 58449 26677025 A  ERROR'
2018-11-27 07:24:37 [DEBUG   ] Got command BAM from MCS: ref #641667
2018-11-27 07:24:37 [INFO    ] Finished processing BAM request
2018-11-27 07:24:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:24:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641667   8 58449 26677029 A  ERROR'
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:24:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:24:37 [DEBUG   ] spawn_process: at 2018-11-27 07:24:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:24:41 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:25:01 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:25:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:25:04 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:25:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:25:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:25:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:25:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:25:04 [WARNING ] (13, 1, 'bot')
2018-11-27 07:25:04 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:25:04 [DEBUG   ] Finished updating board flags in 1.046 seconds
2018-11-27 07:25:17 [DEBUG   ] Got command BAM from MCS: ref #641673
2018-11-27 07:25:17 [INFO    ] Finished processing BAM request
2018-11-27 07:25:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:25:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641673   8 58449 26717029 A  ERROR'
2018-11-27 07:25:17 [DEBUG   ] Got command BAM from MCS: ref #641672
2018-11-27 07:25:17 [INFO    ] Finished processing BAM request
2018-11-27 07:25:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:25:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641672   8 58449 26717033 A  ERROR'
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:25:17 [DEBUG   ] spawn_process: at 2018-11-27 07:25:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:25:21 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:25:41 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:25:48 [DEBUG   ] Got command RPT from MCS: ref #641674
2018-11-27 07:25:48 [DEBUG   ] summary = ERROR
2018-11-27 07:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641674  13 58449 26748974 A  ERRORERROR'
2018-11-27 07:25:48 [DEBUG   ] Got command RPT from MCS: ref #641675
2018-11-27 07:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641675  77 58449 26748975 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:25:57 [DEBUG   ] Got command BAM from MCS: ref #641691
2018-11-27 07:25:57 [INFO    ] Finished processing BAM request
2018-11-27 07:25:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:25:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641691   8 58449 26757029 A  ERROR'
2018-11-27 07:25:57 [DEBUG   ] Got command BAM from MCS: ref #641692
2018-11-27 07:25:57 [INFO    ] Finished processing BAM request
2018-11-27 07:25:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:25:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641692   8 58449 26757033 A  ERROR'
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:25:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:25:57 [DEBUG   ] spawn_process: at 2018-11-27 07:25:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:26:01 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:26:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:26:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:26:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:26:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:26:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:26:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:26:05 [DEBUG   ] Finished updating board flags in 1.557 seconds
2018-11-27 07:26:21 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:26:37 [DEBUG   ] Got command BAM from MCS: ref #641694
2018-11-27 07:26:37 [INFO    ] Finished processing BAM request
2018-11-27 07:26:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:26:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641694   8 58449 26797029 A  ERROR'
2018-11-27 07:26:37 [DEBUG   ] Got command BAM from MCS: ref #641693
2018-11-27 07:26:37 [INFO    ] Finished processing BAM request
2018-11-27 07:26:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:26:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641693   8 58449 26797033 A  ERROR'
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:26:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:26:37 [DEBUG   ] spawn_process: at 2018-11-27 07:26:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:26:41 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:27:01 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:27:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:27:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:27:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-27 07:27:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2018-11-27 07:27:05 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:27:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:27:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:27:05 [DEBUG   ] Finished updating board flags in 1.415 seconds
2018-11-27 07:27:17 [DEBUG   ] Got command BAM from MCS: ref #641696
2018-11-27 07:27:17 [INFO    ] Finished processing BAM request
2018-11-27 07:27:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:27:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641696   8 58449 26837033 A  ERROR'
2018-11-27 07:27:17 [DEBUG   ] Got command BAM from MCS: ref #641695
2018-11-27 07:27:17 [INFO    ] Finished processing BAM request
2018-11-27 07:27:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:27:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641695   8 58449 26837037 A  ERROR'
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:27:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:27:17 [DEBUG   ] spawn_process: at 2018-11-27 07:27:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:27:21 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:27:41 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:27:57 [DEBUG   ] Got command BAM from MCS: ref #641697
2018-11-27 07:27:57 [INFO    ] Finished processing BAM request
2018-11-27 07:27:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:27:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641697   8 58449 26877034 A  ERROR'
2018-11-27 07:27:57 [DEBUG   ] Got command BAM from MCS: ref #641698
2018-11-27 07:27:57 [INFO    ] Finished processing BAM request
2018-11-27 07:27:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:27:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641698   8 58449 26877038 A  ERROR'
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:27:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:27:57 [DEBUG   ] spawn_process: at 2018-11-27 07:27:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:27:59 [DEBUG   ] Got command RPT from MCS: ref #641699
2018-11-27 07:27:59 [DEBUG   ] summary = ERROR
2018-11-27 07:27:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641699  13 58449 26879978 A  ERRORERROR'
2018-11-27 07:28:00 [DEBUG   ] Got command RPT from MCS: ref #641700
2018-11-27 07:28:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:28:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641700  77 58449 26880181 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:28:00 [DEBUG   ] Got command RPT from MCS: ref #641701
2018-11-27 07:28:00 [DEBUG   ] summary = ERROR
2018-11-27 07:28:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641701  13 58449 26880385 A  ERRORERROR'
2018-11-27 07:28:00 [DEBUG   ] Got command RPT from MCS: ref #641702
2018-11-27 07:28:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:28:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641702  77 58449 26880578 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:28:00 [DEBUG   ] Got command RPT from MCS: ref #641703
2018-11-27 07:28:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:28:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641703  12 58449 26880782 A  ERRORB¡ff'
2018-11-27 07:28:01 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 07:28:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 07:28:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:28:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:28:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:28:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:28:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:28:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:28:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:28:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:28:05 [DEBUG   ] Finished updating board flags in 1.464 seconds
2018-11-27 07:28:21 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:28:37 [DEBUG   ] Got command BAM from MCS: ref #641708
2018-11-27 07:28:37 [INFO    ] Finished processing BAM request
2018-11-27 07:28:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:28:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641708   8 58449 26917025 A  ERROR'
2018-11-27 07:28:37 [DEBUG   ] Got command BAM from MCS: ref #641709
2018-11-27 07:28:37 [INFO    ] Finished processing BAM request
2018-11-27 07:28:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:28:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641709   8 58449 26917029 A  ERROR'
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:28:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:28:37 [DEBUG   ] spawn_process: at 2018-11-27 07:28:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:28:41 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:29:01 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:29:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:29:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2018-11-27 07:29:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 1
2018-11-27 07:29:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:29:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:29:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:29:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:29:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:29:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:29:05 [DEBUG   ] Finished updating board flags in 1.318 seconds
2018-11-27 07:29:17 [DEBUG   ] Got command BAM from MCS: ref #641711
2018-11-27 07:29:17 [INFO    ] Finished processing BAM request
2018-11-27 07:29:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:29:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641711   8 58449 26957033 A  ERROR'
2018-11-27 07:29:17 [DEBUG   ] Got command BAM from MCS: ref #641710
2018-11-27 07:29:17 [INFO    ] Finished processing BAM request
2018-11-27 07:29:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:29:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641710   8 58449 26957037 A  ERROR'
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:29:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:29:17 [DEBUG   ] spawn_process: at 2018-11-27 07:29:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:29:21 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 07:29:41 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:29:57 [DEBUG   ] Got command BAM from MCS: ref #641713
2018-11-27 07:29:57 [INFO    ] Finished processing BAM request
2018-11-27 07:29:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:29:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641713   8 58449 26997025 A  ERROR'
2018-11-27 07:29:57 [DEBUG   ] Got command BAM from MCS: ref #641712
2018-11-27 07:29:57 [INFO    ] Finished processing BAM request
2018-11-27 07:29:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:29:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641712   8 58449 26997029 A  ERROR'
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:29:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:29:57 [DEBUG   ] spawn_process: at 2018-11-27 07:29:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:30:01 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 07:30:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:30:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 07:30:05 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:30:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:30:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:30:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:30:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:30:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:30:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:30:05 [DEBUG   ] Finished updating board flags in 1.224 seconds
2018-11-27 07:30:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:30:37 [DEBUG   ] Got command BAM from MCS: ref #641715
2018-11-27 07:30:37 [INFO    ] Finished processing BAM request
2018-11-27 07:30:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:30:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641715   8 58449 27037029 A  ERROR'
2018-11-27 07:30:37 [DEBUG   ] Got command BAM from MCS: ref #641714
2018-11-27 07:30:37 [INFO    ] Finished processing BAM request
2018-11-27 07:30:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:30:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641714   8 58449 27037033 A  ERROR'
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:30:37 [DEBUG   ] spawn_process: at 2018-11-27 07:30:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:30:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:30:48 [DEBUG   ] Got command RPT from MCS: ref #641716
2018-11-27 07:30:48 [DEBUG   ] summary = ERROR
2018-11-27 07:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641716  13 58449 27048957 A  ERRORERROR'
2018-11-27 07:30:48 [DEBUG   ] Got command RPT from MCS: ref #641717
2018-11-27 07:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641717  77 58449 27048958 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:31:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:31:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 07:31:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:31:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:31:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 07:31:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:31:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:31:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:31:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:31:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:31:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:31:05 [DEBUG   ] Finished updating board flags in 1.175 seconds
2018-11-27 07:31:17 [DEBUG   ] Got command BAM from MCS: ref #641737
2018-11-27 07:31:17 [INFO    ] Finished processing BAM request
2018-11-27 07:31:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:31:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641737   8 58449 27077025 A  ERROR'
2018-11-27 07:31:17 [DEBUG   ] Got command BAM from MCS: ref #641738
2018-11-27 07:31:17 [INFO    ] Finished processing BAM request
2018-11-27 07:31:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:31:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641738   8 58449 27077029 A  ERROR'
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:31:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:31:17 [DEBUG   ] spawn_process: at 2018-11-27 07:31:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:31:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:31:41 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:31:57 [DEBUG   ] Got command BAM from MCS: ref #641739
2018-11-27 07:31:57 [INFO    ] Finished processing BAM request
2018-11-27 07:31:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:31:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641739   8 58449 27117033 A  ERROR'
2018-11-27 07:31:57 [DEBUG   ] Got command BAM from MCS: ref #641740
2018-11-27 07:31:57 [INFO    ] Finished processing BAM request
2018-11-27 07:31:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:31:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641740   8 58449 27117037 A  ERROR'
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:31:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:31:57 [DEBUG   ] spawn_process: at 2018-11-27 07:31:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:32:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:32:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:32:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 07:32:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:32:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 07:32:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:32:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-27 07:32:05 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:32:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:32:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:32:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:32:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:32:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:32:05 [DEBUG   ] Finished updating board flags in 1.006 seconds
2018-11-27 07:32:21 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:32:37 [DEBUG   ] Got command BAM from MCS: ref #641742
2018-11-27 07:32:37 [INFO    ] Finished processing BAM request
2018-11-27 07:32:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:32:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641742   8 58449 27157033 A  ERROR'
2018-11-27 07:32:37 [DEBUG   ] Got command BAM from MCS: ref #641741
2018-11-27 07:32:37 [INFO    ] Finished processing BAM request
2018-11-27 07:32:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:32:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641741   8 58449 27157037 A  ERROR'
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:32:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:32:37 [DEBUG   ] spawn_process: at 2018-11-27 07:32:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:32:41 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:33:00 [DEBUG   ] Got command RPT from MCS: ref #641745
2018-11-27 07:33:00 [DEBUG   ] summary = ERROR
2018-11-27 07:33:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641745  13 58449 27180061 A  ERRORERROR'
2018-11-27 07:33:00 [DEBUG   ] Got command RPT from MCS: ref #641746
2018-11-27 07:33:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:33:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641746  77 58449 27180265 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:33:00 [DEBUG   ] Got command RPT from MCS: ref #641747
2018-11-27 07:33:00 [DEBUG   ] summary = ERROR
2018-11-27 07:33:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641747  13 58449 27180470 A  ERRORERROR'
2018-11-27 07:33:00 [DEBUG   ] Got command RPT from MCS: ref #641748
2018-11-27 07:33:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:33:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641748  77 58449 27180665 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:33:00 [DEBUG   ] Got command RPT from MCS: ref #641750
2018-11-27 07:33:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:33:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641750  12 58449 27180869 A  ERRORB¡ff'
2018-11-27 07:33:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:33:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:33:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:33:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:33:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:33:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:33:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:33:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:33:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:33:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:33:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:33:05 [DEBUG   ] Finished updating board flags in 1.019 seconds
2018-11-27 07:33:17 [DEBUG   ] Got command BAM from MCS: ref #641756
2018-11-27 07:33:17 [INFO    ] Finished processing BAM request
2018-11-27 07:33:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:33:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641756   8 58449 27197033 A  ERROR'
2018-11-27 07:33:17 [DEBUG   ] Got command BAM from MCS: ref #641755
2018-11-27 07:33:17 [INFO    ] Finished processing BAM request
2018-11-27 07:33:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:33:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641755   8 58449 27197037 A  ERROR'
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:33:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:33:17 [DEBUG   ] spawn_process: at 2018-11-27 07:33:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:33:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:33:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:33:57 [DEBUG   ] Got command BAM from MCS: ref #641758
2018-11-27 07:33:57 [INFO    ] Finished processing BAM request
2018-11-27 07:33:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:33:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641758   8 58449 27237033 A  ERROR'
2018-11-27 07:33:57 [DEBUG   ] Got command BAM from MCS: ref #641757
2018-11-27 07:33:57 [INFO    ] Finished processing BAM request
2018-11-27 07:33:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:33:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641757   8 58449 27237037 A  ERROR'
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:33:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:33:57 [DEBUG   ] spawn_process: at 2018-11-27 07:33:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:34:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:34:04 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:34:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:34:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2018-11-27 07:34:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:34:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:34:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:34:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:34:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:34:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:34:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:34:05 [DEBUG   ] Finished updating board flags in 1.075 seconds
2018-11-27 07:34:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:34:37 [DEBUG   ] Got command BAM from MCS: ref #641763
2018-11-27 07:34:37 [INFO    ] Finished processing BAM request
2018-11-27 07:34:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:34:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641763   8 58449 27277033 A  ERROR'
2018-11-27 07:34:37 [DEBUG   ] Got command BAM from MCS: ref #641764
2018-11-27 07:34:37 [INFO    ] Finished processing BAM request
2018-11-27 07:34:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:34:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641764   8 58449 27277037 A  ERROR'
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:34:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:34:37 [DEBUG   ] spawn_process: at 2018-11-27 07:34:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:34:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:35:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:35:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:35:06 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:35:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:35:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:35:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:35:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:35:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:35:06 [DEBUG   ] Finished updating board flags in 2.313 seconds
2018-11-27 07:35:17 [DEBUG   ] Got command BAM from MCS: ref #641766
2018-11-27 07:35:17 [INFO    ] Finished processing BAM request
2018-11-27 07:35:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:35:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641766   8 58449 27317029 A  ERROR'
2018-11-27 07:35:17 [DEBUG   ] Got command BAM from MCS: ref #641765
2018-11-27 07:35:17 [INFO    ] Finished processing BAM request
2018-11-27 07:35:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:35:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641765   8 58449 27317033 A  ERROR'
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:35:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:35:17 [DEBUG   ] spawn_process: at 2018-11-27 07:35:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:35:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:35:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:35:48 [DEBUG   ] Got command RPT from MCS: ref #641767
2018-11-27 07:35:48 [DEBUG   ] summary = ERROR
2018-11-27 07:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641767  13 58449 27348970 A  ERRORERROR'
2018-11-27 07:35:48 [DEBUG   ] Got command RPT from MCS: ref #641768
2018-11-27 07:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641768  77 58449 27348970 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:35:57 [DEBUG   ] Got command BAM from MCS: ref #641785
2018-11-27 07:35:57 [INFO    ] Finished processing BAM request
2018-11-27 07:35:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:35:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641785   8 58449 27357025 A  ERROR'
2018-11-27 07:35:57 [DEBUG   ] Got command BAM from MCS: ref #641784
2018-11-27 07:35:57 [INFO    ] Finished processing BAM request
2018-11-27 07:35:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:35:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641784   8 58449 27357029 A  ERROR'
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:35:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:35:57 [DEBUG   ] spawn_process: at 2018-11-27 07:35:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:36:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:36:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:36:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:36:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:36:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:36:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:36:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:36:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:36:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:36:05 [DEBUG   ] Finished updating board flags in 1.243 seconds
2018-11-27 07:36:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:36:37 [DEBUG   ] Got command BAM from MCS: ref #641787
2018-11-27 07:36:37 [INFO    ] Finished processing BAM request
2018-11-27 07:36:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:36:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641787   8 58449 27397025 A  ERROR'
2018-11-27 07:36:37 [DEBUG   ] Got command BAM from MCS: ref #641786
2018-11-27 07:36:37 [INFO    ] Finished processing BAM request
2018-11-27 07:36:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:36:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641786   8 58449 27397029 A  ERROR'
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:36:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:36:37 [DEBUG   ] spawn_process: at 2018-11-27 07:36:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:36:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:37:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:37:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 07:37:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:37:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:37:05 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:37:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:37:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:37:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:37:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:37:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:37:05 [DEBUG   ] Finished updating board flags in 1.226 seconds
2018-11-27 07:37:17 [DEBUG   ] Got command BAM from MCS: ref #641793
2018-11-27 07:37:17 [INFO    ] Finished processing BAM request
2018-11-27 07:37:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:37:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641793   8 58449 27437033 A  ERROR'
2018-11-27 07:37:17 [DEBUG   ] Got command BAM from MCS: ref #641792
2018-11-27 07:37:17 [INFO    ] Finished processing BAM request
2018-11-27 07:37:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:37:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641792   8 58449 27437037 A  ERROR'
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:37:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:37:17 [DEBUG   ] spawn_process: at 2018-11-27 07:37:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:37:21 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:37:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:37:57 [DEBUG   ] Got command BAM from MCS: ref #641794
2018-11-27 07:37:57 [INFO    ] Finished processing BAM request
2018-11-27 07:37:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:37:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641794   8 58449 27477029 A  ERROR'
2018-11-27 07:37:57 [DEBUG   ] Got command BAM from MCS: ref #641795
2018-11-27 07:37:57 [INFO    ] Finished processing BAM request
2018-11-27 07:37:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:37:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641795   8 58449 27477033 A  ERROR'
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:37:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:37:57 [DEBUG   ] spawn_process: at 2018-11-27 07:37:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:38:00 [DEBUG   ] Got command RPT from MCS: ref #641796
2018-11-27 07:38:00 [DEBUG   ] summary = ERROR
2018-11-27 07:38:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641796  13 58449 27480153 A  ERRORERROR'
2018-11-27 07:38:00 [DEBUG   ] Got command RPT from MCS: ref #641797
2018-11-27 07:38:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:38:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641797  77 58449 27480357 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:38:00 [DEBUG   ] Got command RPT from MCS: ref #641798
2018-11-27 07:38:00 [DEBUG   ] summary = ERROR
2018-11-27 07:38:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641798  13 58449 27480557 A  ERRORERROR'
2018-11-27 07:38:00 [DEBUG   ] Got command RPT from MCS: ref #641799
2018-11-27 07:38:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:38:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641799  77 58449 27480749 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:38:00 [DEBUG   ] Got command RPT from MCS: ref #641800
2018-11-27 07:38:00 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:38:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641800  12 58449 27480953 A  ERRORB£ff'
2018-11-27 07:38:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:38:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 07:38:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:38:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:38:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:38:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:38:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:38:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:38:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:38:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:38:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:38:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:38:05 [DEBUG   ] Finished updating board flags in 1.182 seconds
2018-11-27 07:38:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:38:37 [DEBUG   ] Got command BAM from MCS: ref #641802
2018-11-27 07:38:37 [INFO    ] Finished processing BAM request
2018-11-27 07:38:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:38:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641802   8 58449 27517029 A  ERROR'
2018-11-27 07:38:37 [DEBUG   ] Got command BAM from MCS: ref #641801
2018-11-27 07:38:37 [INFO    ] Finished processing BAM request
2018-11-27 07:38:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:38:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641801   8 58449 27517033 A  ERROR'
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:38:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:38:37 [DEBUG   ] spawn_process: at 2018-11-27 07:38:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:38:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:39:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:39:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:39:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:39:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:39:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:39:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:39:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:39:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:39:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:39:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:39:05 [DEBUG   ] Finished updating board flags in 1.032 seconds
2018-11-27 07:39:17 [DEBUG   ] Got command BAM from MCS: ref #641804
2018-11-27 07:39:17 [INFO    ] Finished processing BAM request
2018-11-27 07:39:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:39:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641804   8 58449 27557033 A  ERROR'
2018-11-27 07:39:17 [DEBUG   ] Got command BAM from MCS: ref #641803
2018-11-27 07:39:17 [INFO    ] Finished processing BAM request
2018-11-27 07:39:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:39:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641803   8 58449 27557037 A  ERROR'
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:39:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:39:17 [DEBUG   ] spawn_process: at 2018-11-27 07:39:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:39:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:39:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:39:57 [DEBUG   ] Got command BAM from MCS: ref #641805
2018-11-27 07:39:57 [INFO    ] Finished processing BAM request
2018-11-27 07:39:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:39:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641805   8 58449 27597033 A  ERROR'
2018-11-27 07:39:57 [DEBUG   ] Got command BAM from MCS: ref #641806
2018-11-27 07:39:57 [INFO    ] Finished processing BAM request
2018-11-27 07:39:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:39:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641806   8 58449 27597037 A  ERROR'
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:39:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:39:57 [DEBUG   ] spawn_process: at 2018-11-27 07:39:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:40:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:40:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:40:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:40:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 07:40:06 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:40:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:40:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:40:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:40:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:40:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:40:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:40:06 [DEBUG   ] Finished updating board flags in 1.606 seconds
2018-11-27 07:40:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:40:37 [DEBUG   ] Got command BAM from MCS: ref #641811
2018-11-27 07:40:37 [INFO    ] Finished processing BAM request
2018-11-27 07:40:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:40:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641811   8 58449 27637033 A  ERROR'
2018-11-27 07:40:37 [DEBUG   ] Got command BAM from MCS: ref #641812
2018-11-27 07:40:37 [INFO    ] Finished processing BAM request
2018-11-27 07:40:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:40:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641812   8 58449 27637037 A  ERROR'
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:40:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:40:37 [DEBUG   ] spawn_process: at 2018-11-27 07:40:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:40:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:40:48 [DEBUG   ] Got command RPT from MCS: ref #641813
2018-11-27 07:40:48 [DEBUG   ] summary = ERROR
2018-11-27 07:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641813  13 58449 27648957 A  ERRORERROR'
2018-11-27 07:40:48 [DEBUG   ] Got command RPT from MCS: ref #641814
2018-11-27 07:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641814  77 58449 27648958 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:41:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:41:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 07:41:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:41:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 07:41:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:41:05 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:41:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:41:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:41:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:41:05 [WARNING ] (13, 1, 'bot')
2018-11-27 07:41:05 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:41:05 [DEBUG   ] Finished updating board flags in 0.963 seconds
2018-11-27 07:41:17 [DEBUG   ] Got command BAM from MCS: ref #641831
2018-11-27 07:41:17 [INFO    ] Finished processing BAM request
2018-11-27 07:41:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:41:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641831   8 58449 27677029 A  ERROR'
2018-11-27 07:41:17 [DEBUG   ] Got command BAM from MCS: ref #641830
2018-11-27 07:41:17 [INFO    ] Finished processing BAM request
2018-11-27 07:41:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:41:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641830   8 58449 27677033 A  ERROR'
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:41:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:41:17 [DEBUG   ] spawn_process: at 2018-11-27 07:41:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:41:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:41:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:41:57 [DEBUG   ] Got command BAM from MCS: ref #641832
2018-11-27 07:41:57 [INFO    ] Finished processing BAM request
2018-11-27 07:41:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:41:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641832   8 58449 27717033 A  ERROR'
2018-11-27 07:41:57 [DEBUG   ] Got command BAM from MCS: ref #641833
2018-11-27 07:41:57 [INFO    ] Finished processing BAM request
2018-11-27 07:41:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:41:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641833   8 58449 27717037 A  ERROR'
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:41:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:41:57 [DEBUG   ] spawn_process: at 2018-11-27 07:41:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:42:01 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:42:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:42:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:42:06 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:42:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:42:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:42:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:42:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:42:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:42:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:42:06 [DEBUG   ] Finished updating board flags in 1.360 seconds
2018-11-27 07:42:21 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 07:42:37 [DEBUG   ] Got command BAM from MCS: ref #641835
2018-11-27 07:42:37 [INFO    ] Finished processing BAM request
2018-11-27 07:42:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:42:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641835   8 58449 27757029 A  ERROR'
2018-11-27 07:42:37 [DEBUG   ] Got command BAM from MCS: ref #641834
2018-11-27 07:42:37 [INFO    ] Finished processing BAM request
2018-11-27 07:42:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:42:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641834   8 58449 27757033 A  ERROR'
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:42:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:42:37 [DEBUG   ] spawn_process: at 2018-11-27 07:42:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:42:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:43:00 [DEBUG   ] Got command RPT from MCS: ref #641838
2018-11-27 07:43:00 [DEBUG   ] summary = ERROR
2018-11-27 07:43:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641838  13 58449 27780241 A  ERRORERROR'
2018-11-27 07:43:00 [DEBUG   ] Got command RPT from MCS: ref #641839
2018-11-27 07:43:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:43:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641839  77 58449 27780433 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:43:00 [DEBUG   ] Got command RPT from MCS: ref #641840
2018-11-27 07:43:00 [DEBUG   ] summary = ERROR
2018-11-27 07:43:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641840  13 58449 27780637 A  ERRORERROR'
2018-11-27 07:43:00 [DEBUG   ] Got command RPT from MCS: ref #641842
2018-11-27 07:43:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:43:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641842  77 58449 27780841 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:43:01 [DEBUG   ] Got command RPT from MCS: ref #641843
2018-11-27 07:43:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:43:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641843  12 58449 27781045 A  ERRORB¢ff'
2018-11-27 07:43:01 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:43:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:43:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 07:43:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 5
2018-11-27 07:43:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:43:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 07:43:06 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:43:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:43:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:43:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:43:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:43:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:43:06 [DEBUG   ] Finished updating board flags in 1.211 seconds
2018-11-27 07:43:17 [DEBUG   ] Got command BAM from MCS: ref #641852
2018-11-27 07:43:17 [INFO    ] Finished processing BAM request
2018-11-27 07:43:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:43:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641852   8 58449 27797029 A  ERROR'
2018-11-27 07:43:17 [DEBUG   ] Got command BAM from MCS: ref #641853
2018-11-27 07:43:17 [INFO    ] Finished processing BAM request
2018-11-27 07:43:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:43:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641853   8 58449 27797033 A  ERROR'
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:43:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:43:17 [DEBUG   ] spawn_process: at 2018-11-27 07:43:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:43:21 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 07:43:41 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 07:43:57 [DEBUG   ] Got command BAM from MCS: ref #641854
2018-11-27 07:43:57 [INFO    ] Finished processing BAM request
2018-11-27 07:43:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:43:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641854   8 58449 27837025 A  ERROR'
2018-11-27 07:43:57 [DEBUG   ] Got command BAM from MCS: ref #641855
2018-11-27 07:43:57 [INFO    ] Finished processing BAM request
2018-11-27 07:43:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:43:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641855   8 58449 27837029 A  ERROR'
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:43:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:43:57 [DEBUG   ] spawn_process: at 2018-11-27 07:43:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:44:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 07:44:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:44:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:44:06 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:44:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:44:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:44:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:44:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:44:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:44:06 [DEBUG   ] Finished updating board flags in 1.193 seconds
2018-11-27 07:44:21 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 07:44:37 [DEBUG   ] Got command BAM from MCS: ref #641857
2018-11-27 07:44:37 [INFO    ] Finished processing BAM request
2018-11-27 07:44:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:44:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641857   8 58449 27877033 A  ERROR'
2018-11-27 07:44:37 [DEBUG   ] Got command BAM from MCS: ref #641856
2018-11-27 07:44:37 [INFO    ] Finished processing BAM request
2018-11-27 07:44:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:44:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641856   8 58449 27877037 A  ERROR'
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:44:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:44:37 [DEBUG   ] spawn_process: at 2018-11-27 07:44:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:44:41 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 07:45:01 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:45:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:45:06 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:45:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:45:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:45:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:45:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:45:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:45:06 [DEBUG   ] Finished updating board flags in 1.008 seconds
2018-11-27 07:45:17 [DEBUG   ] Got command BAM from MCS: ref #641858
2018-11-27 07:45:17 [INFO    ] Finished processing BAM request
2018-11-27 07:45:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:45:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641858   8 58449 27917033 A  ERROR'
2018-11-27 07:45:17 [DEBUG   ] Got command BAM from MCS: ref #641859
2018-11-27 07:45:17 [INFO    ] Finished processing BAM request
2018-11-27 07:45:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:45:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641859   8 58449 27917037 A  ERROR'
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:45:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:45:17 [DEBUG   ] spawn_process: at 2018-11-27 07:45:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:45:21 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:45:41 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:45:48 [DEBUG   ] Got command RPT from MCS: ref #641860
2018-11-27 07:45:48 [DEBUG   ] summary = ERROR
2018-11-27 07:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641860  13 58449 27948957 A  ERRORERROR'
2018-11-27 07:45:48 [DEBUG   ] Got command RPT from MCS: ref #641861
2018-11-27 07:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641861  77 58449 27948958 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:45:57 [DEBUG   ] Got command BAM from MCS: ref #641877
2018-11-27 07:45:57 [INFO    ] Finished processing BAM request
2018-11-27 07:45:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:45:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641877   8 58449 27957025 A  ERROR'
2018-11-27 07:45:57 [DEBUG   ] Got command BAM from MCS: ref #641878
2018-11-27 07:45:57 [INFO    ] Finished processing BAM request
2018-11-27 07:45:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:45:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641878   8 58449 27957029 A  ERROR'
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:45:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:45:57 [DEBUG   ] spawn_process: at 2018-11-27 07:45:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:46:01 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:46:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:46:06 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:46:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:46:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:46:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:46:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:46:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:46:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:46:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:46:06 [DEBUG   ] Finished updating board flags in 1.005 seconds
2018-11-27 07:46:21 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:46:37 [DEBUG   ] Got command BAM from MCS: ref #641884
2018-11-27 07:46:37 [INFO    ] Finished processing BAM request
2018-11-27 07:46:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:46:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641884   8 58449 27997029 A  ERROR'
2018-11-27 07:46:37 [DEBUG   ] Got command BAM from MCS: ref #641883
2018-11-27 07:46:37 [INFO    ] Finished processing BAM request
2018-11-27 07:46:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:46:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641883   8 58449 27997033 A  ERROR'
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:46:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:46:37 [DEBUG   ] spawn_process: at 2018-11-27 07:46:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:46:41 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:47:01 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 07:47:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 07:47:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 5
2018-11-27 07:47:05 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:47:06 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:47:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:47:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:47:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:47:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:47:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:47:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:47:06 [DEBUG   ] Finished updating board flags in 0.985 seconds
2018-11-27 07:47:17 [DEBUG   ] Got command BAM from MCS: ref #641885
2018-11-27 07:47:17 [INFO    ] Finished processing BAM request
2018-11-27 07:47:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:47:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641885   8 58449 28037033 A  ERROR'
2018-11-27 07:47:17 [DEBUG   ] Got command BAM from MCS: ref #641886
2018-11-27 07:47:17 [INFO    ] Finished processing BAM request
2018-11-27 07:47:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:47:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641886   8 58449 28037041 A  ERROR'
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:47:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:47:17 [DEBUG   ] spawn_process: at 2018-11-27 07:47:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:47:21 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 07:47:42 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 07:47:57 [DEBUG   ] Got command BAM from MCS: ref #641888
2018-11-27 07:47:57 [INFO    ] Finished processing BAM request
2018-11-27 07:47:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:47:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641888   8 58449 28077025 A  ERROR'
2018-11-27 07:47:57 [DEBUG   ] Got command BAM from MCS: ref #641887
2018-11-27 07:47:57 [INFO    ] Finished processing BAM request
2018-11-27 07:47:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:47:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641887   8 58449 28077029 A  ERROR'
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:47:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:47:57 [DEBUG   ] spawn_process: at 2018-11-27 07:47:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:48:00 [DEBUG   ] Got command RPT from MCS: ref #641889
2018-11-27 07:48:00 [DEBUG   ] summary = ERROR
2018-11-27 07:48:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641889  13 58449 28080321 A  ERRORERROR'
2018-11-27 07:48:00 [DEBUG   ] Got command RPT from MCS: ref #641890
2018-11-27 07:48:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:48:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641890  77 58449 28080525 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:48:00 [DEBUG   ] Got command RPT from MCS: ref #641891
2018-11-27 07:48:00 [DEBUG   ] summary = ERROR
2018-11-27 07:48:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641891  13 58449 28080729 A  ERRORERROR'
2018-11-27 07:48:00 [DEBUG   ] Got command RPT from MCS: ref #641892
2018-11-27 07:48:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:48:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641892  77 58449 28080921 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:48:01 [DEBUG   ] Got command RPT from MCS: ref #641893
2018-11-27 07:48:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:48:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641893  12 58449 28081121 A  ERRORB£ff'
2018-11-27 07:48:02 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 07:48:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 07:48:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:48:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:48:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 07:48:07 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:48:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:48:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:48:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:48:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:48:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:48:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:48:07 [DEBUG   ] Finished updating board flags in 1.814 seconds
2018-11-27 07:48:22 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 07:48:37 [DEBUG   ] Got command BAM from MCS: ref #641895
2018-11-27 07:48:37 [INFO    ] Finished processing BAM request
2018-11-27 07:48:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:48:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641895   8 58449 28117025 A  ERROR'
2018-11-27 07:48:37 [DEBUG   ] Got command BAM from MCS: ref #641894
2018-11-27 07:48:37 [INFO    ] Finished processing BAM request
2018-11-27 07:48:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:48:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641894   8 58449 28117029 A  ERROR'
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:48:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:48:37 [DEBUG   ] spawn_process: at 2018-11-27 07:48:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:48:42 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 07:49:02 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 07:49:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:49:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:49:07 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:49:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:49:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:49:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:49:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:49:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:49:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:49:07 [DEBUG   ] Finished updating board flags in 1.793 seconds
2018-11-27 07:49:17 [DEBUG   ] Got command BAM from MCS: ref #641900
2018-11-27 07:49:17 [INFO    ] Finished processing BAM request
2018-11-27 07:49:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:49:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641900   8 58449 28157033 A  ERROR'
2018-11-27 07:49:17 [DEBUG   ] Got command BAM from MCS: ref #641901
2018-11-27 07:49:17 [INFO    ] Finished processing BAM request
2018-11-27 07:49:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:49:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641901   8 58449 28157037 A  ERROR'
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:49:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:49:17 [DEBUG   ] spawn_process: at 2018-11-27 07:49:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:49:22 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 07:49:42 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:49:57 [DEBUG   ] Got command BAM from MCS: ref #641903
2018-11-27 07:49:57 [INFO    ] Finished processing BAM request
2018-11-27 07:49:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:49:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641903   8 58449 28197033 A  ERROR'
2018-11-27 07:49:57 [DEBUG   ] Got command BAM from MCS: ref #641902
2018-11-27 07:49:57 [INFO    ] Finished processing BAM request
2018-11-27 07:49:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:49:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641902   8 58449 28197037 A  ERROR'
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:49:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:49:57 [DEBUG   ] spawn_process: at 2018-11-27 07:49:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:50:02 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:50:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:50:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 07:50:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 07:50:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:50:06 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:50:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:50:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:50:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:50:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:50:06 [WARNING ] (13, 1, 'bot')
2018-11-27 07:50:06 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:50:06 [DEBUG   ] Finished updating board flags in 1.053 seconds
2018-11-27 07:50:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:50:37 [DEBUG   ] Got command BAM from MCS: ref #641905
2018-11-27 07:50:37 [INFO    ] Finished processing BAM request
2018-11-27 07:50:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:50:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641905   8 58449 28237025 A  ERROR'
2018-11-27 07:50:37 [DEBUG   ] Got command BAM from MCS: ref #641904
2018-11-27 07:50:37 [INFO    ] Finished processing BAM request
2018-11-27 07:50:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:50:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641904   8 58449 28237029 A  ERROR'
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:50:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:50:37 [DEBUG   ] spawn_process: at 2018-11-27 07:50:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:50:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:50:48 [DEBUG   ] Got command RPT from MCS: ref #641906
2018-11-27 07:50:48 [DEBUG   ] summary = ERROR
2018-11-27 07:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641906  13 58449 28248965 A  ERRORERROR'
2018-11-27 07:50:48 [DEBUG   ] Got command RPT from MCS: ref #641907
2018-11-27 07:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641907  77 58449 28248966 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:51:02 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:51:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:51:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:51:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:51:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:51:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:51:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:51:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:51:07 [DEBUG   ] Finished updating board flags in 1.622 seconds
2018-11-27 07:51:17 [DEBUG   ] Got command BAM from MCS: ref #641924
2018-11-27 07:51:17 [INFO    ] Finished processing BAM request
2018-11-27 07:51:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:51:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641924   8 58449 28277033 A  ERROR'
2018-11-27 07:51:17 [DEBUG   ] Got command BAM from MCS: ref #641923
2018-11-27 07:51:17 [INFO    ] Finished processing BAM request
2018-11-27 07:51:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:51:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641923   8 58449 28277037 A  ERROR'
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:51:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:51:17 [DEBUG   ] spawn_process: at 2018-11-27 07:51:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:51:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:51:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:51:57 [DEBUG   ] Got command BAM from MCS: ref #641925
2018-11-27 07:51:57 [INFO    ] Finished processing BAM request
2018-11-27 07:51:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:51:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641925   8 58449 28317029 A  ERROR'
2018-11-27 07:51:57 [DEBUG   ] Got command BAM from MCS: ref #641926
2018-11-27 07:51:57 [INFO    ] Finished processing BAM request
2018-11-27 07:51:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:51:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641926   8 58449 28317033 A  ERROR'
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:51:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:51:57 [DEBUG   ] spawn_process: at 2018-11-27 07:51:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:52:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:52:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:52:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:52:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:52:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:52:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:52:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:52:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:52:07 [DEBUG   ] Finished updating board flags in 1.525 seconds
2018-11-27 07:52:22 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:52:37 [DEBUG   ] Got command BAM from MCS: ref #641931
2018-11-27 07:52:37 [INFO    ] Finished processing BAM request
2018-11-27 07:52:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:52:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641931   8 58449 28357029 A  ERROR'
2018-11-27 07:52:37 [DEBUG   ] Got command BAM from MCS: ref #641932
2018-11-27 07:52:37 [INFO    ] Finished processing BAM request
2018-11-27 07:52:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:52:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641932   8 58449 28357033 A  ERROR'
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:52:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:52:37 [DEBUG   ] spawn_process: at 2018-11-27 07:52:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:52:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:53:00 [DEBUG   ] Got command RPT from MCS: ref #641935
2018-11-27 07:53:00 [DEBUG   ] summary = ERROR
2018-11-27 07:53:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641935  13 58449 28380409 A  ERRORERROR'
2018-11-27 07:53:00 [DEBUG   ] Got command RPT from MCS: ref #641936
2018-11-27 07:53:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:53:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641936  77 58449 28380613 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:53:00 [DEBUG   ] Got command RPT from MCS: ref #641937
2018-11-27 07:53:00 [DEBUG   ] summary = ERROR
2018-11-27 07:53:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641937  13 58449 28380817 A  ERRORERROR'
2018-11-27 07:53:01 [DEBUG   ] Got command RPT from MCS: ref #641939
2018-11-27 07:53:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:53:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641939  77 58449 28381009 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:53:01 [DEBUG   ] Got command RPT from MCS: ref #641941
2018-11-27 07:53:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:53:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641941  12 58449 28381213 A  ERRORB¢ff'
2018-11-27 07:53:02 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:53:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 07:53:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 07:53:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:53:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:53:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:53:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:53:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:53:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:53:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:53:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:53:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:53:07 [DEBUG   ] Finished updating board flags in 1.408 seconds
2018-11-27 07:53:17 [DEBUG   ] Got command BAM from MCS: ref #641945
2018-11-27 07:53:17 [INFO    ] Finished processing BAM request
2018-11-27 07:53:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:53:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641945   8 58449 28397033 A  ERROR'
2018-11-27 07:53:17 [DEBUG   ] Got command BAM from MCS: ref #641946
2018-11-27 07:53:17 [INFO    ] Finished processing BAM request
2018-11-27 07:53:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:53:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641946   8 58449 28397037 A  ERROR'
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:53:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:53:17 [DEBUG   ] spawn_process: at 2018-11-27 07:53:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:53:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:53:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:53:57 [DEBUG   ] Got command BAM from MCS: ref #641948
2018-11-27 07:53:57 [INFO    ] Finished processing BAM request
2018-11-27 07:53:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:53:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641948   8 58449 28437029 A  ERROR'
2018-11-27 07:53:57 [DEBUG   ] Got command BAM from MCS: ref #641947
2018-11-27 07:53:57 [INFO    ] Finished processing BAM request
2018-11-27 07:53:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:53:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641947   8 58449 28437033 A  ERROR'
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:53:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:53:57 [DEBUG   ] spawn_process: at 2018-11-27 07:53:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:54:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:54:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:54:07 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:54:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 07:54:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:54:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:54:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:54:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:54:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:54:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:54:07 [DEBUG   ] Finished updating board flags in 1.413 seconds
2018-11-27 07:54:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:54:37 [DEBUG   ] Got command BAM from MCS: ref #641949
2018-11-27 07:54:37 [INFO    ] Finished processing BAM request
2018-11-27 07:54:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:54:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641949   8 58449 28477029 A  ERROR'
2018-11-27 07:54:37 [DEBUG   ] Got command BAM from MCS: ref #641950
2018-11-27 07:54:37 [INFO    ] Finished processing BAM request
2018-11-27 07:54:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:54:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641950   8 58449 28477033 A  ERROR'
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 07:54:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 07:54:37 [DEBUG   ] spawn_process: at 2018-11-27 07:54:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 07:54:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:55:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:55:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 07:55:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:55:07 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 07:55:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:55:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:55:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:55:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:55:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:55:07 [DEBUG   ] Finished updating board flags in 1.397 seconds
2018-11-27 07:55:17 [DEBUG   ] Got command BAM from MCS: ref #641955
2018-11-27 07:55:17 [INFO    ] Finished processing BAM request
2018-11-27 07:55:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:55:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641955   8 58449 28517025 A  ERROR'
2018-11-27 07:55:17 [DEBUG   ] Got command BAM from MCS: ref #641956
2018-11-27 07:55:17 [INFO    ] Finished processing BAM request
2018-11-27 07:55:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:55:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641956   8 58449 28517029 A  ERROR'
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:55:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:55:17 [DEBUG   ] spawn_process: at 2018-11-27 07:55:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:55:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:55:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:55:48 [DEBUG   ] Got command RPT from MCS: ref #641957
2018-11-27 07:55:48 [DEBUG   ] summary = ERROR
2018-11-27 07:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641957  13 58449 28548961 A  ERRORERROR'
2018-11-27 07:55:48 [DEBUG   ] Got command RPT from MCS: ref #641958
2018-11-27 07:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641958  77 58449 28548962 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:55:57 [DEBUG   ] Got command BAM from MCS: ref #641974
2018-11-27 07:55:57 [INFO    ] Finished processing BAM request
2018-11-27 07:55:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:55:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641974   8 58449 28557029 A  ERROR'
2018-11-27 07:55:57 [DEBUG   ] Got command BAM from MCS: ref #641975
2018-11-27 07:55:57 [INFO    ] Finished processing BAM request
2018-11-27 07:55:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:55:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641975   8 58449 28557033 A  ERROR'
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:55:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:55:57 [DEBUG   ] spawn_process: at 2018-11-27 07:55:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:56:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:56:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:56:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 07:56:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:56:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:56:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:56:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:56:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:56:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:56:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:56:07 [DEBUG   ] Finished updating board flags in 1.280 seconds
2018-11-27 07:56:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:56:37 [DEBUG   ] Got command BAM from MCS: ref #641977
2018-11-27 07:56:37 [INFO    ] Finished processing BAM request
2018-11-27 07:56:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:56:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641977   8 58449 28597025 A  ERROR'
2018-11-27 07:56:37 [DEBUG   ] Got command BAM from MCS: ref #641976
2018-11-27 07:56:37 [INFO    ] Finished processing BAM request
2018-11-27 07:56:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:56:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641976   8 58449 28597029 A  ERROR'
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 07:56:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 07:56:37 [DEBUG   ] spawn_process: at 2018-11-27 07:56:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 07:56:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:57:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:57:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 07:57:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 07:57:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:57:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 07:57:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:57:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 07:57:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:57:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:57:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:57:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:57:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:57:07 [DEBUG   ] Finished updating board flags in 1.110 seconds
2018-11-27 07:57:17 [DEBUG   ] Got command BAM from MCS: ref #641978
2018-11-27 07:57:17 [INFO    ] Finished processing BAM request
2018-11-27 07:57:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:57:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641978   8 58449 28637033 A  ERROR'
2018-11-27 07:57:17 [DEBUG   ] Got command BAM from MCS: ref #641979
2018-11-27 07:57:17 [INFO    ] Finished processing BAM request
2018-11-27 07:57:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:57:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641979   8 58449 28637037 A  ERROR'
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 07:57:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 07:57:17 [DEBUG   ] spawn_process: at 2018-11-27 07:57:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 07:57:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:57:42 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 07:57:57 [DEBUG   ] Got command BAM from MCS: ref #641981
2018-11-27 07:57:57 [INFO    ] Finished processing BAM request
2018-11-27 07:57:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:57:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641981   8 58449 28677033 A  ERROR'
2018-11-27 07:57:57 [DEBUG   ] Got command BAM from MCS: ref #641980
2018-11-27 07:57:57 [INFO    ] Finished processing BAM request
2018-11-27 07:57:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:57:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641980   8 58449 28677037 A  ERROR'
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 07:57:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 07:57:57 [DEBUG   ] spawn_process: at 2018-11-27 07:57:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 07:58:00 [DEBUG   ] Got command RPT from MCS: ref #641982
2018-11-27 07:58:00 [DEBUG   ] summary = ERROR
2018-11-27 07:58:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641982  13 58449 28680493 A  ERRORERROR'
2018-11-27 07:58:00 [DEBUG   ] Got command RPT from MCS: ref #641983
2018-11-27 07:58:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:58:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641983  77 58449 28680697 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:58:00 [DEBUG   ] Got command RPT from MCS: ref #641984
2018-11-27 07:58:00 [DEBUG   ] summary = ERROR
2018-11-27 07:58:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641984  13 58449 28680901 A  ERRORERROR'
2018-11-27 07:58:01 [DEBUG   ] Got command RPT from MCS: ref #641985
2018-11-27 07:58:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 07:58:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641985  77 58449 28681093 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 07:58:01 [DEBUG   ] Got command RPT from MCS: ref #641986
2018-11-27 07:58:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 07:58:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641986  12 58449 28681297 A  ERRORB¢ff'
2018-11-27 07:58:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:58:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 07:58:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:58:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 07:58:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:58:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:58:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:58:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:58:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:58:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:58:07 [DEBUG   ] Finished updating board flags in 1.013 seconds
2018-11-27 07:58:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:58:37 [DEBUG   ] Got command BAM from MCS: ref #641991
2018-11-27 07:58:37 [INFO    ] Finished processing BAM request
2018-11-27 07:58:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:58:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641991   8 58449 28717033 A  ERROR'
2018-11-27 07:58:37 [DEBUG   ] Got command BAM from MCS: ref #641992
2018-11-27 07:58:37 [INFO    ] Finished processing BAM request
2018-11-27 07:58:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:58:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641992   8 58449 28717037 A  ERROR'
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:58:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:58:37 [DEBUG   ] spawn_process: at 2018-11-27 07:58:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:58:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:59:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 07:59:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 07:59:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 07:59:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 07:59:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:59:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:59:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:59:07 [WARNING ] (13, 1, 'bot')
2018-11-27 07:59:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 07:59:07 [DEBUG   ] Finished updating board flags in 1.009 seconds
2018-11-27 07:59:17 [DEBUG   ] Got command BAM from MCS: ref #641994
2018-11-27 07:59:17 [INFO    ] Finished processing BAM request
2018-11-27 07:59:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:59:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641994   8 58449 28757025 A  ERROR'
2018-11-27 07:59:17 [DEBUG   ] Got command BAM from MCS: ref #641993
2018-11-27 07:59:17 [INFO    ] Finished processing BAM request
2018-11-27 07:59:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:59:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641993   8 58449 28757029 A  ERROR'
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:59:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:59:17 [DEBUG   ] spawn_process: at 2018-11-27 07:59:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 07:59:22 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 07:59:42 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 07:59:57 [DEBUG   ] Got command BAM from MCS: ref #641996
2018-11-27 07:59:57 [INFO    ] Finished processing BAM request
2018-11-27 07:59:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:59:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641996   8 58449 28797033 A  ERROR'
2018-11-27 07:59:57 [DEBUG   ] Got command BAM from MCS: ref #641995
2018-11-27 07:59:57 [INFO    ] Finished processing BAM request
2018-11-27 07:59:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 07:59:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641995   8 58449 28797037 A  ERROR'
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 07:59:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 07:59:57 [DEBUG   ] spawn_process: at 2018-11-27 07:59:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:00:02 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 08:00:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 08:00:06 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:00:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:00:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 08:00:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:00:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:00:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:00:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:00:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:00:07 [DEBUG   ] Finished updating board flags in 0.988 seconds
2018-11-27 08:00:22 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 08:00:37 [DEBUG   ] Got command BAM from MCS: ref #641998
2018-11-27 08:00:37 [INFO    ] Finished processing BAM request
2018-11-27 08:00:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:00:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641998   8 58449 28837030 A  ERROR'
2018-11-27 08:00:37 [DEBUG   ] Got command BAM from MCS: ref #641997
2018-11-27 08:00:37 [INFO    ] Finished processing BAM request
2018-11-27 08:00:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:00:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   641997   8 58449 28837034 A  ERROR'
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:00:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:00:37 [DEBUG   ] spawn_process: at 2018-11-27 08:00:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:00:42 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:00:48 [DEBUG   ] Got command RPT from MCS: ref #641999
2018-11-27 08:00:48 [DEBUG   ] summary = ERROR
2018-11-27 08:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   641999  13 58449 28848961 A  ERRORERROR'
2018-11-27 08:00:48 [DEBUG   ] Got command RPT from MCS: ref #642000
2018-11-27 08:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642000  77 58449 28848962 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:01:02 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 08:01:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:01:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 08:01:08 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:01:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 08:01:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:01:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:01:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:01:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:01:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:01:08 [DEBUG   ] Finished updating board flags in 1.825 seconds
2018-11-27 08:01:18 [DEBUG   ] Got command BAM from MCS: ref #642020
2018-11-27 08:01:18 [INFO    ] Finished processing BAM request
2018-11-27 08:01:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:01:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642020   8 58449 28878025 A  ERROR'
2018-11-27 08:01:18 [DEBUG   ] Got command BAM from MCS: ref #642021
2018-11-27 08:01:18 [INFO    ] Finished processing BAM request
2018-11-27 08:01:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:01:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642021   8 58449 28878029 A  ERROR'
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 08:01:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 08:01:18 [DEBUG   ] spawn_process: at 2018-11-27 08:01:18 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 08:01:22 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:01:42 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:01:59 [DEBUG   ] Got command BAM from MCS: ref #642023
2018-11-27 08:01:59 [INFO    ] Finished processing BAM request
2018-11-27 08:01:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:01:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642023   8 58449 28919029 A  ERROR'
2018-11-27 08:01:59 [DEBUG   ] Got command BAM from MCS: ref #642022
2018-11-27 08:01:59 [INFO    ] Finished processing BAM request
2018-11-27 08:01:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:01:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642022   8 58449 28919033 A  ERROR'
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:01:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:01:59 [DEBUG   ] spawn_process: at 2018-11-27 08:01:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:02:02 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:02:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:02:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 08:02:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:02:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2018-11-27 08:02:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:02:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:02:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:02:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:02:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:02:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:02:07 [DEBUG   ] Finished updating board flags in 1.219 seconds
2018-11-27 08:02:22 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:02:40 [DEBUG   ] Got command BAM from MCS: ref #642024
2018-11-27 08:02:40 [INFO    ] Finished processing BAM request
2018-11-27 08:02:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:02:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642024   8 58449 28960025 A  ERROR'
2018-11-27 08:02:40 [DEBUG   ] Got command BAM from MCS: ref #642025
2018-11-27 08:02:40 [INFO    ] Finished processing BAM request
2018-11-27 08:02:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:02:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642025   8 58449 28960029 A  ERROR'
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:02:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:02:40 [DEBUG   ] spawn_process: at 2018-11-27 08:02:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:02:42 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:03:00 [DEBUG   ] Got command RPT from MCS: ref #642028
2018-11-27 08:03:00 [DEBUG   ] summary = ERROR
2018-11-27 08:03:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642028  13 58449 28980585 A  ERRORERROR'
2018-11-27 08:03:00 [DEBUG   ] Got command RPT from MCS: ref #642029
2018-11-27 08:03:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:03:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642029  77 58449 28980777 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:03:00 [DEBUG   ] Got command RPT from MCS: ref #642030
2018-11-27 08:03:00 [DEBUG   ] summary = ERROR
2018-11-27 08:03:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642030  13 58449 28980981 A  ERRORERROR'
2018-11-27 08:03:01 [DEBUG   ] Got command RPT from MCS: ref #642032
2018-11-27 08:03:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:03:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642032  77 58449 28981185 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:03:01 [DEBUG   ] Got command RPT from MCS: ref #642033
2018-11-27 08:03:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:03:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642033  12 58449 28981389 A  ERRORB¢ff'
2018-11-27 08:03:02 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:03:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:03:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:03:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:03:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:03:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:03:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:03:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:03:07 [DEBUG   ] Finished updating board flags in 1.229 seconds
2018-11-27 08:03:21 [DEBUG   ] Got command BAM from MCS: ref #642038
2018-11-27 08:03:21 [INFO    ] Finished processing BAM request
2018-11-27 08:03:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:03:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642038   8 58449 29001029 A  ERROR'
2018-11-27 08:03:21 [DEBUG   ] Got command BAM from MCS: ref #642039
2018-11-27 08:03:21 [INFO    ] Finished processing BAM request
2018-11-27 08:03:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:03:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642039   8 58449 29001033 A  ERROR'
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:03:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:03:21 [DEBUG   ] spawn_process: at 2018-11-27 08:03:21 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:03:22 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:03:42 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:04:02 [DEBUG   ] Got command BAM from MCS: ref #642040
2018-11-27 08:04:02 [INFO    ] Finished processing BAM request
2018-11-27 08:04:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:04:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642040   8 58449 29042029 A  ERROR'
2018-11-27 08:04:02 [DEBUG   ] Got command BAM from MCS: ref #642041
2018-11-27 08:04:02 [INFO    ] Finished processing BAM request
2018-11-27 08:04:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:04:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642041   8 58449 29042033 A  ERROR'
2018-11-27 08:04:02 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 08:04:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 08:04:02 [DEBUG   ] spawn_process: at 2018-11-27 08:04:02 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 08:04:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:04:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:04:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:04:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:04:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 08:04:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 08:04:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:04:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:04:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:04:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:04:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:04:07 [DEBUG   ] Finished updating board flags in 1.173 seconds
2018-11-27 08:04:22 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:04:42 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:04:43 [DEBUG   ] Got command BAM from MCS: ref #642047
2018-11-27 08:04:43 [INFO    ] Finished processing BAM request
2018-11-27 08:04:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:04:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642047   8 58449 29083025 A  ERROR'
2018-11-27 08:04:43 [DEBUG   ] Got command BAM from MCS: ref #642046
2018-11-27 08:04:43 [INFO    ] Finished processing BAM request
2018-11-27 08:04:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:04:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642046   8 58449 29083029 A  ERROR'
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:04:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:04:43 [DEBUG   ] spawn_process: at 2018-11-27 08:04:43 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:05:02 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:05:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:05:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:05:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 08:05:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:05:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:05:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:05:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:05:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:05:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:05:07 [DEBUG   ] Finished updating board flags in 1.081 seconds
2018-11-27 08:05:22 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:05:24 [DEBUG   ] Got command BAM from MCS: ref #642048
2018-11-27 08:05:24 [INFO    ] Finished processing BAM request
2018-11-27 08:05:24 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:05:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642048   8 58449 29124033 A  ERROR'
2018-11-27 08:05:24 [DEBUG   ] Got command BAM from MCS: ref #642049
2018-11-27 08:05:24 [INFO    ] Finished processing BAM request
2018-11-27 08:05:24 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:05:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642049   8 58449 29124037 A  ERROR'
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:05:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:05:24 [DEBUG   ] spawn_process: at 2018-11-27 08:05:24 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:05:42 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:05:48 [DEBUG   ] Got command RPT from MCS: ref #642050
2018-11-27 08:05:48 [DEBUG   ] summary = ERROR
2018-11-27 08:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642050  13 58449 29148965 A  ERRORERROR'
2018-11-27 08:05:48 [DEBUG   ] Got command RPT from MCS: ref #642051
2018-11-27 08:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642051  77 58449 29148966 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:06:02 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 08:06:05 [DEBUG   ] Got command BAM from MCS: ref #642068
2018-11-27 08:06:05 [INFO    ] Finished processing BAM request
2018-11-27 08:06:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:06:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642068   8 58449 29165025 A  ERROR'
2018-11-27 08:06:05 [DEBUG   ] Got command BAM from MCS: ref #642067
2018-11-27 08:06:05 [INFO    ] Finished processing BAM request
2018-11-27 08:06:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:06:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642067   8 58449 29165029 A  ERROR'
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:06:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:06:05 [DEBUG   ] spawn_process: at 2018-11-27 08:06:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:06:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:06:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:06:07 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:06:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:06:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:06:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:06:07 [WARNING ] (13, 1, 'bot')
2018-11-27 08:06:07 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:06:07 [DEBUG   ] Finished updating board flags in 0.932 seconds
2018-11-27 08:06:22 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 08:06:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:06:46 [DEBUG   ] Got command BAM from MCS: ref #642070
2018-11-27 08:06:46 [INFO    ] Finished processing BAM request
2018-11-27 08:06:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:06:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642070   8 58449 29206025 A  ERROR'
2018-11-27 08:06:46 [DEBUG   ] Got command BAM from MCS: ref #642069
2018-11-27 08:06:46 [INFO    ] Finished processing BAM request
2018-11-27 08:06:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:06:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642069   8 58449 29206029 A  ERROR'
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:06:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:06:46 [DEBUG   ] spawn_process: at 2018-11-27 08:06:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:07:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:07:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 08:07:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:07:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 08:07:08 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:07:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:07:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:07:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:07:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:07:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:07:08 [DEBUG   ] Finished updating board flags in 1.353 seconds
2018-11-27 08:07:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:07:27 [DEBUG   ] Got command BAM from MCS: ref #642075
2018-11-27 08:07:27 [INFO    ] Finished processing BAM request
2018-11-27 08:07:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:07:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642075   8 58449 29247033 A  ERROR'
2018-11-27 08:07:27 [DEBUG   ] Got command BAM from MCS: ref #642076
2018-11-27 08:07:27 [INFO    ] Finished processing BAM request
2018-11-27 08:07:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:07:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642076   8 58449 29247037 A  ERROR'
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:07:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:07:27 [DEBUG   ] spawn_process: at 2018-11-27 08:07:27 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:07:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:08:00 [DEBUG   ] Got command RPT from MCS: ref #642077
2018-11-27 08:08:00 [DEBUG   ] summary = ERROR
2018-11-27 08:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642077  13 58449 29280665 A  ERRORERROR'
2018-11-27 08:08:00 [DEBUG   ] Got command RPT from MCS: ref #642078
2018-11-27 08:08:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642078  77 58449 29280869 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:08:01 [DEBUG   ] Got command RPT from MCS: ref #642079
2018-11-27 08:08:01 [DEBUG   ] summary = ERROR
2018-11-27 08:08:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642079  13 58449 29281073 A  ERRORERROR'
2018-11-27 08:08:01 [DEBUG   ] Got command RPT from MCS: ref #642080
2018-11-27 08:08:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:08:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642080  77 58449 29281265 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:08:01 [DEBUG   ] Got command RPT from MCS: ref #642081
2018-11-27 08:08:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:08:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642081  12 58449 29281469 A  ERRORB¢ff'
2018-11-27 08:08:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:08:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:08:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 08:08:08 [DEBUG   ] Got command BAM from MCS: ref #642083
2018-11-27 08:08:08 [INFO    ] Finished processing BAM request
2018-11-27 08:08:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:08:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642083   8 58449 29288033 A  ERROR'
2018-11-27 08:08:08 [DEBUG   ] Got command BAM from MCS: ref #642082
2018-11-27 08:08:08 [INFO    ] Finished processing BAM request
2018-11-27 08:08:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:08:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642082   8 58449 29288037 A  ERROR'
2018-11-27 08:08:08 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:08:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:08:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:08:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:08:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:08:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:08:08 [DEBUG   ] Finished updating board flags in 1.221 seconds
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 08:08:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 08:08:08 [DEBUG   ] spawn_process: at 2018-11-27 08:08:08 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 08:08:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:08:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:08:49 [DEBUG   ] Got command BAM from MCS: ref #642085
2018-11-27 08:08:49 [INFO    ] Finished processing BAM request
2018-11-27 08:08:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:08:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642085   8 58449 29329025 A  ERROR'
2018-11-27 08:08:49 [DEBUG   ] Got command BAM from MCS: ref #642084
2018-11-27 08:08:49 [INFO    ] Finished processing BAM request
2018-11-27 08:08:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:08:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642084   8 58449 29329029 A  ERROR'
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:08:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:08:49 [DEBUG   ] spawn_process: at 2018-11-27 08:08:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:09:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:09:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:09:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:09:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 08:09:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 1
2018-11-27 08:09:08 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:09:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 08:09:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:09:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:09:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:09:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:09:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:09:08 [DEBUG   ] Finished updating board flags in 1.209 seconds
2018-11-27 08:09:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:09:30 [DEBUG   ] Got command BAM from MCS: ref #642086
2018-11-27 08:09:30 [INFO    ] Finished processing BAM request
2018-11-27 08:09:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:09:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642086   8 58449 29370029 A  ERROR'
2018-11-27 08:09:30 [DEBUG   ] Got command BAM from MCS: ref #642087
2018-11-27 08:09:30 [INFO    ] Finished processing BAM request
2018-11-27 08:09:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:09:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642087   8 58449 29370033 A  ERROR'
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:09:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:09:30 [DEBUG   ] spawn_process: at 2018-11-27 08:09:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:09:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:10:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:10:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:10:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 08:10:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:10:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 08:10:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:10:08 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:10:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 08:10:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:10:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:10:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:10:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:10:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:10:08 [DEBUG   ] Finished updating board flags in 1.168 seconds
2018-11-27 08:10:11 [DEBUG   ] Got command BAM from MCS: ref #642089
2018-11-27 08:10:11 [INFO    ] Finished processing BAM request
2018-11-27 08:10:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:10:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642089   8 58449 29411025 A  ERROR'
2018-11-27 08:10:11 [DEBUG   ] Got command BAM from MCS: ref #642088
2018-11-27 08:10:11 [INFO    ] Finished processing BAM request
2018-11-27 08:10:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:10:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642088   8 58449 29411029 A  ERROR'
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:10:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:10:11 [DEBUG   ] spawn_process: at 2018-11-27 08:10:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:10:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:10:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:10:48 [DEBUG   ] Got command RPT from MCS: ref #642094
2018-11-27 08:10:48 [DEBUG   ] summary = ERROR
2018-11-27 08:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642094  13 58449 29448969 A  ERRORERROR'
2018-11-27 08:10:48 [DEBUG   ] Got command RPT from MCS: ref #642095
2018-11-27 08:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642095  77 58449 29448970 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:10:52 [DEBUG   ] Got command BAM from MCS: ref #642111
2018-11-27 08:10:52 [INFO    ] Finished processing BAM request
2018-11-27 08:10:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:10:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642111   8 58449 29452033 A  ERROR'
2018-11-27 08:10:52 [DEBUG   ] Got command BAM from MCS: ref #642112
2018-11-27 08:10:52 [INFO    ] Finished processing BAM request
2018-11-27 08:10:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:10:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642112   8 58449 29452037 A  ERROR'
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 08:10:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 08:10:52 [DEBUG   ] spawn_process: at 2018-11-27 08:10:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 08:11:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:11:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:11:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 1
2018-11-27 08:11:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 08:11:07 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:11:08 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:11:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:11:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:11:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:11:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:11:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:11:08 [DEBUG   ] Finished updating board flags in 1.045 seconds
2018-11-27 08:11:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:11:33 [DEBUG   ] Got command BAM from MCS: ref #642113
2018-11-27 08:11:33 [INFO    ] Finished processing BAM request
2018-11-27 08:11:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:11:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642113   8 58449 29493025 A  ERROR'
2018-11-27 08:11:33 [DEBUG   ] Got command BAM from MCS: ref #642114
2018-11-27 08:11:33 [INFO    ] Finished processing BAM request
2018-11-27 08:11:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:11:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642114   8 58449 29493029 A  ERROR'
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:11:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:11:33 [DEBUG   ] spawn_process: at 2018-11-27 08:11:33 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:11:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:12:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:12:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:12:09 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:12:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:12:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:12:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:12:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:12:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:12:09 [DEBUG   ] Finished updating board flags in 2.005 seconds
2018-11-27 08:12:14 [DEBUG   ] Got command BAM from MCS: ref #642116
2018-11-27 08:12:14 [INFO    ] Finished processing BAM request
2018-11-27 08:12:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:12:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642116   8 58449 29534029 A  ERROR'
2018-11-27 08:12:14 [DEBUG   ] Got command BAM from MCS: ref #642115
2018-11-27 08:12:14 [INFO    ] Finished processing BAM request
2018-11-27 08:12:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:12:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642115   8 58449 29534033 A  ERROR'
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:12:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:12:14 [DEBUG   ] spawn_process: at 2018-11-27 08:12:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:12:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:12:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:12:55 [DEBUG   ] Got command BAM from MCS: ref #642119
2018-11-27 08:12:55 [INFO    ] Finished processing BAM request
2018-11-27 08:12:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:12:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642119   8 58449 29575029 A  ERROR'
2018-11-27 08:12:55 [DEBUG   ] Got command BAM from MCS: ref #642120
2018-11-27 08:12:55 [INFO    ] Finished processing BAM request
2018-11-27 08:12:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:12:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642120   8 58449 29575033 A  ERROR'
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:12:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:12:55 [DEBUG   ] spawn_process: at 2018-11-27 08:12:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:13:00 [DEBUG   ] Got command RPT from MCS: ref #642121
2018-11-27 08:13:00 [DEBUG   ] summary = ERROR
2018-11-27 08:13:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642121  13 58449 29580757 A  ERRORERROR'
2018-11-27 08:13:00 [DEBUG   ] Got command RPT from MCS: ref #642122
2018-11-27 08:13:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:13:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642122  77 58449 29580949 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:13:01 [DEBUG   ] Got command RPT from MCS: ref #642123
2018-11-27 08:13:01 [DEBUG   ] summary = ERROR
2018-11-27 08:13:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642123  13 58449 29581153 A  ERRORERROR'
2018-11-27 08:13:01 [DEBUG   ] Got command RPT from MCS: ref #642125
2018-11-27 08:13:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:13:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642125  77 58449 29581357 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:13:01 [DEBUG   ] Got command RPT from MCS: ref #642126
2018-11-27 08:13:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:13:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642126  12 58449 29581561 A  ERRORB¢ff'
2018-11-27 08:13:02 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 08:13:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:13:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 08:13:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:13:08 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:13:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 08:13:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:13:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:13:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:13:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:13:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:13:08 [DEBUG   ] Finished updating board flags in 1.410 seconds
2018-11-27 08:13:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:13:36 [DEBUG   ] Got command BAM from MCS: ref #642136
2018-11-27 08:13:36 [INFO    ] Finished processing BAM request
2018-11-27 08:13:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:13:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642136   8 58449 29616025 A  ERROR'
2018-11-27 08:13:36 [DEBUG   ] Got command BAM from MCS: ref #642135
2018-11-27 08:13:36 [INFO    ] Finished processing BAM request
2018-11-27 08:13:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:13:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642135   8 58449 29616029 A  ERROR'
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:13:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:13:36 [DEBUG   ] spawn_process: at 2018-11-27 08:13:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:13:42 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 08:14:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:14:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 08:14:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:14:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 08:14:08 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:14:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:14:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:14:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:14:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:14:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:14:08 [DEBUG   ] Finished updating board flags in 1.406 seconds
2018-11-27 08:14:17 [DEBUG   ] Got command BAM from MCS: ref #642138
2018-11-27 08:14:17 [INFO    ] Finished processing BAM request
2018-11-27 08:14:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:14:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642138   8 58449 29657025 A  ERROR'
2018-11-27 08:14:17 [DEBUG   ] Got command BAM from MCS: ref #642137
2018-11-27 08:14:17 [INFO    ] Finished processing BAM request
2018-11-27 08:14:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:14:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642137   8 58449 29657029 A  ERROR'
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:14:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:14:17 [DEBUG   ] spawn_process: at 2018-11-27 08:14:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:14:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:14:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:14:58 [DEBUG   ] Got command BAM from MCS: ref #642139
2018-11-27 08:14:58 [INFO    ] Finished processing BAM request
2018-11-27 08:14:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:14:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642139   8 58449 29698025 A  ERROR'
2018-11-27 08:14:58 [DEBUG   ] Got command BAM from MCS: ref #642140
2018-11-27 08:14:58 [INFO    ] Finished processing BAM request
2018-11-27 08:14:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:14:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642140   8 58449 29698029 A  ERROR'
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:14:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:14:58 [DEBUG   ] spawn_process: at 2018-11-27 08:14:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:15:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:15:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:15:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 08:15:08 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:15:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:15:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:15:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:15:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:15:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:15:08 [DEBUG   ] Finished updating board flags in 1.432 seconds
2018-11-27 08:15:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:15:39 [DEBUG   ] Got command BAM from MCS: ref #642142
2018-11-27 08:15:39 [INFO    ] Finished processing BAM request
2018-11-27 08:15:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:15:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642142   8 58449 29739033 A  ERROR'
2018-11-27 08:15:39 [DEBUG   ] Got command BAM from MCS: ref #642141
2018-11-27 08:15:39 [INFO    ] Finished processing BAM request
2018-11-27 08:15:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:15:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642141   8 58449 29739037 A  ERROR'
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:15:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:15:39 [DEBUG   ] spawn_process: at 2018-11-27 08:15:39 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:15:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:15:48 [DEBUG   ] Got command RPT from MCS: ref #642143
2018-11-27 08:15:48 [DEBUG   ] summary = ERROR
2018-11-27 08:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642143  13 58449 29748961 A  ERRORERROR'
2018-11-27 08:15:48 [DEBUG   ] Got command RPT from MCS: ref #642144
2018-11-27 08:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642144  77 58449 29748962 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:16:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:16:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 08:16:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:16:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:16:08 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:16:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:16:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:16:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:16:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:16:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:16:08 [DEBUG   ] Finished updating board flags in 1.361 seconds
2018-11-27 08:16:20 [DEBUG   ] Got command BAM from MCS: ref #642164
2018-11-27 08:16:20 [INFO    ] Finished processing BAM request
2018-11-27 08:16:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:16:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642164   8 58449 29780029 A  ERROR'
2018-11-27 08:16:20 [DEBUG   ] Got command BAM from MCS: ref #642165
2018-11-27 08:16:20 [INFO    ] Finished processing BAM request
2018-11-27 08:16:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:16:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642165   8 58449 29780033 A  ERROR'
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 08:16:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 08:16:20 [DEBUG   ] spawn_process: at 2018-11-27 08:16:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 08:16:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:16:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:17:01 [DEBUG   ] Got command BAM from MCS: ref #642166
2018-11-27 08:17:01 [INFO    ] Finished processing BAM request
2018-11-27 08:17:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:17:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642166   8 58449 29821033 A  ERROR'
2018-11-27 08:17:01 [DEBUG   ] Got command BAM from MCS: ref #642167
2018-11-27 08:17:01 [INFO    ] Finished processing BAM request
2018-11-27 08:17:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:17:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642167   8 58449 29821037 A  ERROR'
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:17:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:17:01 [DEBUG   ] spawn_process: at 2018-11-27 08:17:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:17:02 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 08:17:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 08:17:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:17:08 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:17:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:17:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:17:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:17:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:17:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:17:08 [DEBUG   ] Finished updating board flags in 1.218 seconds
2018-11-27 08:17:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:17:42 [DEBUG   ] Got command BAM from MCS: ref #642168
2018-11-27 08:17:42 [INFO    ] Finished processing BAM request
2018-11-27 08:17:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:17:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642168   8 58449 29862029 A  ERROR'
2018-11-27 08:17:42 [DEBUG   ] Got command BAM from MCS: ref #642169
2018-11-27 08:17:42 [INFO    ] Finished processing BAM request
2018-11-27 08:17:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:17:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642169   8 58449 29862033 A  ERROR'
2018-11-27 08:17:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:17:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:17:42 [DEBUG   ] spawn_process: at 2018-11-27 08:17:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:18:00 [DEBUG   ] Got command RPT from MCS: ref #642170
2018-11-27 08:18:00 [DEBUG   ] summary = ERROR
2018-11-27 08:18:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642170  13 58449 29880841 A  ERRORERROR'
2018-11-27 08:18:01 [DEBUG   ] Got command RPT from MCS: ref #642171
2018-11-27 08:18:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:18:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642171  77 58449 29881045 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:18:01 [DEBUG   ] Got command RPT from MCS: ref #642172
2018-11-27 08:18:01 [DEBUG   ] summary = ERROR
2018-11-27 08:18:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642172  13 58449 29881249 A  ERRORERROR'
2018-11-27 08:18:01 [DEBUG   ] Got command RPT from MCS: ref #642173
2018-11-27 08:18:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:18:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642173  77 58449 29881441 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:18:01 [DEBUG   ] Got command RPT from MCS: ref #642174
2018-11-27 08:18:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:18:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642174  12 58449 29881645 A  ERRORB¡ff'
2018-11-27 08:18:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:18:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:18:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 08:18:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2018-11-27 08:18:09 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:18:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 08:18:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:18:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:18:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:18:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:18:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:18:09 [DEBUG   ] Finished updating board flags in 1.787 seconds
2018-11-27 08:18:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:18:23 [DEBUG   ] Got command BAM from MCS: ref #642175
2018-11-27 08:18:23 [INFO    ] Finished processing BAM request
2018-11-27 08:18:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:18:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642175   8 58449 29903033 A  ERROR'
2018-11-27 08:18:23 [DEBUG   ] Got command BAM from MCS: ref #642176
2018-11-27 08:18:23 [INFO    ] Finished processing BAM request
2018-11-27 08:18:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:18:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642176   8 58449 29903037 A  ERROR'
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 08:18:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 08:18:23 [DEBUG   ] spawn_process: at 2018-11-27 08:18:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 08:18:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:19:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:19:04 [DEBUG   ] Got command BAM from MCS: ref #642177
2018-11-27 08:19:04 [INFO    ] Finished processing BAM request
2018-11-27 08:19:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:19:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642177   8 58449 29944033 A  ERROR'
2018-11-27 08:19:04 [DEBUG   ] Got command BAM from MCS: ref #642178
2018-11-27 08:19:04 [INFO    ] Finished processing BAM request
2018-11-27 08:19:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:19:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642178   8 58449 29944037 A  ERROR'
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:19:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:19:04 [DEBUG   ] spawn_process: at 2018-11-27 08:19:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:19:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 08:19:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 08:19:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:19:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 08:19:08 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:19:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 08:19:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:19:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:19:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:19:08 [WARNING ] (13, 1, 'bot')
2018-11-27 08:19:08 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:19:08 [DEBUG   ] Finished updating board flags in 0.987 seconds
2018-11-27 08:19:22 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:19:42 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:19:45 [DEBUG   ] Got command BAM from MCS: ref #642183
2018-11-27 08:19:45 [INFO    ] Finished processing BAM request
2018-11-27 08:19:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:19:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642183   8 58449 29985029 A  ERROR'
2018-11-27 08:19:45 [DEBUG   ] Got command BAM from MCS: ref #642184
2018-11-27 08:19:45 [INFO    ] Finished processing BAM request
2018-11-27 08:19:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:19:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642184   8 58449 29985033 A  ERROR'
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:19:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:19:45 [DEBUG   ] spawn_process: at 2018-11-27 08:19:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:20:02 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:20:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 3
2018-11-27 08:20:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:20:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:20:09 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:20:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:20:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:20:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:20:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:20:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:20:09 [DEBUG   ] Finished updating board flags in 1.371 seconds
2018-11-27 08:20:22 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 08:20:26 [DEBUG   ] Got command BAM from MCS: ref #642185
2018-11-27 08:20:26 [INFO    ] Finished processing BAM request
2018-11-27 08:20:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:20:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642185   8 58449 30026025 A  ERROR'
2018-11-27 08:20:26 [DEBUG   ] Got command BAM from MCS: ref #642186
2018-11-27 08:20:26 [INFO    ] Finished processing BAM request
2018-11-27 08:20:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:20:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642186   8 58449 30026029 A  ERROR'
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:20:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:20:26 [DEBUG   ] spawn_process: at 2018-11-27 08:20:26 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:20:43 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 08:20:48 [DEBUG   ] Got command RPT from MCS: ref #642187
2018-11-27 08:20:48 [DEBUG   ] summary = ERROR
2018-11-27 08:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642187  13 58449 30048965 A  ERRORERROR'
2018-11-27 08:20:48 [DEBUG   ] Got command RPT from MCS: ref #642188
2018-11-27 08:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642188  77 58449 30048966 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:21:03 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 08:21:07 [DEBUG   ] Got command BAM from MCS: ref #642205
2018-11-27 08:21:07 [INFO    ] Finished processing BAM request
2018-11-27 08:21:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:21:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642205   8 58449 30067033 A  ERROR'
2018-11-27 08:21:07 [DEBUG   ] Got command BAM from MCS: ref #642204
2018-11-27 08:21:07 [INFO    ] Finished processing BAM request
2018-11-27 08:21:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:21:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642204   8 58449 30067037 A  ERROR'
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:21:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:21:07 [DEBUG   ] spawn_process: at 2018-11-27 08:21:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:21:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:21:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:21:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 08:21:09 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:21:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:21:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:21:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:21:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:21:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:21:09 [DEBUG   ] Finished updating board flags in 1.189 seconds
2018-11-27 08:21:23 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 08:21:43 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:21:48 [DEBUG   ] Got command BAM from MCS: ref #642207
2018-11-27 08:21:48 [INFO    ] Finished processing BAM request
2018-11-27 08:21:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:21:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642207   8 58449 30108029 A  ERROR'
2018-11-27 08:21:48 [DEBUG   ] Got command BAM from MCS: ref #642206
2018-11-27 08:21:48 [INFO    ] Finished processing BAM request
2018-11-27 08:21:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:21:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642206   8 58449 30108033 A  ERROR'
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:21:48 [DEBUG   ] spawn_process: at 2018-11-27 08:21:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:22:03 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 08:22:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:22:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:22:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 1
2018-11-27 08:22:09 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:22:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 08:22:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:22:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:22:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:22:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:22:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:22:09 [DEBUG   ] Finished updating board flags in 1.015 seconds
2018-11-27 08:22:23 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:22:29 [DEBUG   ] Got command BAM from MCS: ref #642212
2018-11-27 08:22:29 [INFO    ] Finished processing BAM request
2018-11-27 08:22:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:22:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642212   8 58449 30149029 A  ERROR'
2018-11-27 08:22:29 [DEBUG   ] Got command BAM from MCS: ref #642213
2018-11-27 08:22:29 [INFO    ] Finished processing BAM request
2018-11-27 08:22:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:22:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642213   8 58449 30149033 A  ERROR'
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 08:22:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 08:22:29 [DEBUG   ] spawn_process: at 2018-11-27 08:22:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 08:22:43 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:23:00 [DEBUG   ] Got command RPT from MCS: ref #642216
2018-11-27 08:23:00 [DEBUG   ] summary = ERROR
2018-11-27 08:23:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642216  13 58449 30180925 A  ERRORERROR'
2018-11-27 08:23:01 [DEBUG   ] Got command RPT from MCS: ref #642217
2018-11-27 08:23:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:23:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642217  77 58449 30181129 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:23:01 [DEBUG   ] Got command RPT from MCS: ref #642218
2018-11-27 08:23:01 [DEBUG   ] summary = ERROR
2018-11-27 08:23:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642218  13 58449 30181333 A  ERRORERROR'
2018-11-27 08:23:01 [DEBUG   ] Got command RPT from MCS: ref #642220
2018-11-27 08:23:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:23:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642220  77 58449 30181525 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:23:01 [DEBUG   ] Got command RPT from MCS: ref #642221
2018-11-27 08:23:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:23:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642221  12 58449 30181729 A  ERRORB¢ff'
2018-11-27 08:23:03 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:23:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:23:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:23:08 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:23:09 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:23:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 08:23:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:23:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:23:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:23:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:23:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:23:09 [DEBUG   ] Finished updating board flags in 0.998 seconds
2018-11-27 08:23:11 [DEBUG   ] Got command BAM from MCS: ref #642226
2018-11-27 08:23:11 [INFO    ] Finished processing BAM request
2018-11-27 08:23:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:23:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642226   8 58449 30191029 A  ERROR'
2018-11-27 08:23:11 [DEBUG   ] Got command BAM from MCS: ref #642227
2018-11-27 08:23:11 [INFO    ] Finished processing BAM request
2018-11-27 08:23:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:23:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642227   8 58449 30191033 A  ERROR'
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:23:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:23:11 [DEBUG   ] spawn_process: at 2018-11-27 08:23:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:23:23 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 08:23:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:23:53 [DEBUG   ] Got command BAM from MCS: ref #642228
2018-11-27 08:23:53 [INFO    ] Finished processing BAM request
2018-11-27 08:23:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:23:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642228   8 58449 30233025 A  ERROR'
2018-11-27 08:23:53 [DEBUG   ] Got command BAM from MCS: ref #642229
2018-11-27 08:23:53 [INFO    ] Finished processing BAM request
2018-11-27 08:23:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:23:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642229   8 58449 30233029 A  ERROR'
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:23:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:23:53 [DEBUG   ] spawn_process: at 2018-11-27 08:23:53 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:24:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:24:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:24:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:24:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:24:10 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:24:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 08:24:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:24:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:24:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:24:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:24:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:24:10 [DEBUG   ] Finished updating board flags in 1.811 seconds
2018-11-27 08:24:23 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 08:24:35 [DEBUG   ] Got command BAM from MCS: ref #642231
2018-11-27 08:24:35 [INFO    ] Finished processing BAM request
2018-11-27 08:24:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:24:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642231   8 58449 30275029 A  ERROR'
2018-11-27 08:24:35 [DEBUG   ] Got command BAM from MCS: ref #642230
2018-11-27 08:24:35 [INFO    ] Finished processing BAM request
2018-11-27 08:24:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:24:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642230   8 58449 30275033 A  ERROR'
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:24:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:24:35 [DEBUG   ] spawn_process: at 2018-11-27 08:24:35 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:24:43 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 08:25:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:25:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:25:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:25:10 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:25:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:25:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:25:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:25:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:25:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:25:10 [DEBUG   ] Finished updating board flags in 1.858 seconds
2018-11-27 08:25:17 [DEBUG   ] Got command BAM from MCS: ref #642236
2018-11-27 08:25:17 [INFO    ] Finished processing BAM request
2018-11-27 08:25:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:25:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642236   8 58449 30317025 A  ERROR'
2018-11-27 08:25:17 [DEBUG   ] Got command BAM from MCS: ref #642237
2018-11-27 08:25:17 [INFO    ] Finished processing BAM request
2018-11-27 08:25:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:25:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642237   8 58449 30317029 A  ERROR'
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:25:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:25:17 [DEBUG   ] spawn_process: at 2018-11-27 08:25:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:25:23 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 08:25:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:25:48 [DEBUG   ] Got command RPT from MCS: ref #642238
2018-11-27 08:25:48 [DEBUG   ] summary = ERROR
2018-11-27 08:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642238  13 58449 30348965 A  ERRORERROR'
2018-11-27 08:25:48 [DEBUG   ] Got command RPT from MCS: ref #642239
2018-11-27 08:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642239  77 58449 30348966 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:25:59 [DEBUG   ] Got command BAM from MCS: ref #642256
2018-11-27 08:25:59 [INFO    ] Finished processing BAM request
2018-11-27 08:25:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:25:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642256   8 58449 30359025 A  ERROR'
2018-11-27 08:25:59 [DEBUG   ] Got command BAM from MCS: ref #642255
2018-11-27 08:25:59 [INFO    ] Finished processing BAM request
2018-11-27 08:25:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:25:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642255   8 58449 30359029 A  ERROR'
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:25:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:25:59 [DEBUG   ] spawn_process: at 2018-11-27 08:25:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:26:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:26:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:26:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-27 08:26:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 08:26:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 08:26:10 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:26:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:26:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:26:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:26:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:26:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:26:10 [DEBUG   ] Finished updating board flags in 1.768 seconds
2018-11-27 08:26:23 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 08:26:41 [DEBUG   ] Got command BAM from MCS: ref #642258
2018-11-27 08:26:41 [INFO    ] Finished processing BAM request
2018-11-27 08:26:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:26:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642258   8 58449 30401033 A  ERROR'
2018-11-27 08:26:41 [DEBUG   ] Got command BAM from MCS: ref #642257
2018-11-27 08:26:41 [INFO    ] Finished processing BAM request
2018-11-27 08:26:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:26:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642257   8 58449 30401037 A  ERROR'
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:26:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:26:41 [DEBUG   ] spawn_process: at 2018-11-27 08:26:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:26:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:27:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:27:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:27:09 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:27:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:27:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:27:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:27:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:27:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:27:09 [DEBUG   ] Finished updating board flags in 1.038 seconds
2018-11-27 08:27:23 [DEBUG   ] Got command BAM from MCS: ref #642259
2018-11-27 08:27:23 [INFO    ] Finished processing BAM request
2018-11-27 08:27:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:27:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642259   8 58449 30443029 A  ERROR'
2018-11-27 08:27:23 [DEBUG   ] Got command BAM from MCS: ref #642260
2018-11-27 08:27:23 [INFO    ] Finished processing BAM request
2018-11-27 08:27:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:27:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642260   8 58449 30443033 A  ERROR'
2018-11-27 08:27:23 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:27:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:27:23 [DEBUG   ] spawn_process: at 2018-11-27 08:27:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:27:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:28:01 [DEBUG   ] Got command RPT from MCS: ref #642261
2018-11-27 08:28:01 [DEBUG   ] summary = ERROR
2018-11-27 08:28:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642261  13 58449 30481005 A  ERRORERROR'
2018-11-27 08:28:01 [DEBUG   ] Got command RPT from MCS: ref #642262
2018-11-27 08:28:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:28:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642262  77 58449 30481209 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:28:01 [DEBUG   ] Got command RPT from MCS: ref #642263
2018-11-27 08:28:01 [DEBUG   ] summary = ERROR
2018-11-27 08:28:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642263  13 58449 30481413 A  ERRORERROR'
2018-11-27 08:28:01 [DEBUG   ] Got command RPT from MCS: ref #642264
2018-11-27 08:28:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:28:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642264  77 58449 30481617 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:28:01 [DEBUG   ] Got command RPT from MCS: ref #642265
2018-11-27 08:28:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:28:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642265  12 58449 30481809 A  ERRORB¢ff'
2018-11-27 08:28:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:28:05 [DEBUG   ] Got command BAM from MCS: ref #642266
2018-11-27 08:28:05 [INFO    ] Finished processing BAM request
2018-11-27 08:28:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:28:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642266   8 58449 30485033 A  ERROR'
2018-11-27 08:28:05 [DEBUG   ] Got command BAM from MCS: ref #642267
2018-11-27 08:28:05 [INFO    ] Finished processing BAM request
2018-11-27 08:28:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:28:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642267   8 58449 30485037 A  ERROR'
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 08:28:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 08:28:05 [DEBUG   ] spawn_process: at 2018-11-27 08:28:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 08:28:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 08:28:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:28:09 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:28:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:28:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:28:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:28:09 [WARNING ] (13, 1, 'bot')
2018-11-27 08:28:09 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:28:09 [DEBUG   ] Finished updating board flags in 0.989 seconds
2018-11-27 08:28:23 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 08:28:43 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:28:47 [DEBUG   ] Got command BAM from MCS: ref #642273
2018-11-27 08:28:47 [INFO    ] Finished processing BAM request
2018-11-27 08:28:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:28:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642273   8 58449 30527029 A  ERROR'
2018-11-27 08:28:47 [DEBUG   ] Got command BAM from MCS: ref #642272
2018-11-27 08:28:47 [INFO    ] Finished processing BAM request
2018-11-27 08:28:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:28:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642272   8 58449 30527033 A  ERROR'
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:28:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:28:47 [DEBUG   ] spawn_process: at 2018-11-27 08:28:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:29:03 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:29:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:29:10 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:29:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:29:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:29:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:29:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:29:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:29:10 [DEBUG   ] Finished updating board flags in 1.374 seconds
2018-11-27 08:29:23 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:29:29 [DEBUG   ] Got command BAM from MCS: ref #642275
2018-11-27 08:29:29 [INFO    ] Finished processing BAM request
2018-11-27 08:29:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:29:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642275   8 58449 30569029 A  ERROR'
2018-11-27 08:29:29 [DEBUG   ] Got command BAM from MCS: ref #642274
2018-11-27 08:29:29 [INFO    ] Finished processing BAM request
2018-11-27 08:29:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:29:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642274   8 58449 30569033 A  ERROR'
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:29:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:29:29 [DEBUG   ] spawn_process: at 2018-11-27 08:29:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:29:43 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:30:03 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:30:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:30:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:30:10 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:30:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 08:30:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:30:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:30:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:30:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:30:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:30:10 [DEBUG   ] Finished updating board flags in 1.181 seconds
2018-11-27 08:30:11 [DEBUG   ] Got command BAM from MCS: ref #642277
2018-11-27 08:30:11 [INFO    ] Finished processing BAM request
2018-11-27 08:30:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:30:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642277   8 58449 30611025 A  ERROR'
2018-11-27 08:30:11 [DEBUG   ] Got command BAM from MCS: ref #642276
2018-11-27 08:30:11 [INFO    ] Finished processing BAM request
2018-11-27 08:30:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:30:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642276   8 58449 30611029 A  ERROR'
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:30:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:30:11 [DEBUG   ] spawn_process: at 2018-11-27 08:30:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:30:23 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:30:43 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:30:48 [DEBUG   ] Got command RPT from MCS: ref #642278
2018-11-27 08:30:48 [DEBUG   ] summary = ERROR
2018-11-27 08:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642278  13 58449 30648977 A  ERRORERROR'
2018-11-27 08:30:48 [DEBUG   ] Got command RPT from MCS: ref #642279
2018-11-27 08:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642279  77 58449 30648978 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:30:53 [DEBUG   ] Got command BAM from MCS: ref #642295
2018-11-27 08:30:53 [INFO    ] Finished processing BAM request
2018-11-27 08:30:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:30:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642295   8 58449 30653025 A  ERROR'
2018-11-27 08:30:53 [DEBUG   ] Got command BAM from MCS: ref #642296
2018-11-27 08:30:53 [INFO    ] Finished processing BAM request
2018-11-27 08:30:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:30:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642296   8 58449 30653029 A  ERROR'
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:30:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:30:53 [DEBUG   ] spawn_process: at 2018-11-27 08:30:53 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:31:03 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:31:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:31:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:31:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:31:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 08:31:10 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:31:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:31:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:31:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:31:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:31:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:31:10 [DEBUG   ] Finished updating board flags in 1.019 seconds
2018-11-27 08:31:23 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 08:31:35 [DEBUG   ] Got command BAM from MCS: ref #642302
2018-11-27 08:31:35 [INFO    ] Finished processing BAM request
2018-11-27 08:31:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:31:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642302   8 58449 30695029 A  ERROR'
2018-11-27 08:31:35 [DEBUG   ] Got command BAM from MCS: ref #642301
2018-11-27 08:31:35 [INFO    ] Finished processing BAM request
2018-11-27 08:31:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:31:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642301   8 58449 30695033 A  ERROR'
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:31:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:31:35 [DEBUG   ] spawn_process: at 2018-11-27 08:31:35 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:31:43 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 08:32:03 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 08:32:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:32:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:32:09 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 08:32:10 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:32:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:32:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:32:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:32:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:32:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:32:10 [DEBUG   ] Finished updating board flags in 1.055 seconds
2018-11-27 08:32:17 [DEBUG   ] Got command BAM from MCS: ref #642303
2018-11-27 08:32:17 [INFO    ] Finished processing BAM request
2018-11-27 08:32:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:32:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642303   8 58449 30737025 A  ERROR'
2018-11-27 08:32:17 [DEBUG   ] Got command BAM from MCS: ref #642304
2018-11-27 08:32:17 [INFO    ] Finished processing BAM request
2018-11-27 08:32:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:32:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642304   8 58449 30737029 A  ERROR'
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:32:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:32:17 [DEBUG   ] spawn_process: at 2018-11-27 08:32:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:32:23 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:32:43 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:32:59 [DEBUG   ] Got command BAM from MCS: ref #642308
2018-11-27 08:32:59 [INFO    ] Finished processing BAM request
2018-11-27 08:32:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:32:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642308   8 58449 30779025 A  ERROR'
2018-11-27 08:32:59 [DEBUG   ] Got command BAM from MCS: ref #642307
2018-11-27 08:32:59 [INFO    ] Finished processing BAM request
2018-11-27 08:32:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:32:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642307   8 58449 30779029 A  ERROR'
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:32:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:32:59 [DEBUG   ] spawn_process: at 2018-11-27 08:32:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:33:01 [DEBUG   ] Got command RPT from MCS: ref #642309
2018-11-27 08:33:01 [DEBUG   ] summary = ERROR
2018-11-27 08:33:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642309  13 58449 30781093 A  ERRORERROR'
2018-11-27 08:33:01 [DEBUG   ] Got command RPT from MCS: ref #642310
2018-11-27 08:33:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:33:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642310  77 58449 30781297 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:33:01 [DEBUG   ] Got command RPT from MCS: ref #642311
2018-11-27 08:33:01 [DEBUG   ] summary = ERROR
2018-11-27 08:33:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642311  13 58449 30781501 A  ERRORERROR'
2018-11-27 08:33:01 [DEBUG   ] Got command RPT from MCS: ref #642313
2018-11-27 08:33:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:33:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642313  77 58449 30781705 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:33:01 [DEBUG   ] Got command RPT from MCS: ref #642314
2018-11-27 08:33:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:33:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642314  12 58449 30781897 A  ERRORB£ff'
2018-11-27 08:33:03 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:33:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:33:11 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:33:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:33:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:33:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:33:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:33:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:33:11 [DEBUG   ] Finished updating board flags in 1.995 seconds
2018-11-27 08:33:23 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:33:41 [DEBUG   ] Got command BAM from MCS: ref #642319
2018-11-27 08:33:41 [INFO    ] Finished processing BAM request
2018-11-27 08:33:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:33:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642319   8 58449 30821025 A  ERROR'
2018-11-27 08:33:41 [DEBUG   ] Got command BAM from MCS: ref #642320
2018-11-27 08:33:41 [INFO    ] Finished processing BAM request
2018-11-27 08:33:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:33:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642320   8 58449 30821029 A  ERROR'
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:33:41 [DEBUG   ] spawn_process: at 2018-11-27 08:33:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:33:43 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:34:03 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:34:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 08:34:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:34:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:34:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 08:34:11 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:34:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:34:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:34:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:34:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:34:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:34:11 [DEBUG   ] Finished updating board flags in 1.737 seconds
2018-11-27 08:34:23 [DEBUG   ] Got command BAM from MCS: ref #642326
2018-11-27 08:34:23 [INFO    ] Finished processing BAM request
2018-11-27 08:34:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:34:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642326   8 58449 30863033 A  ERROR'
2018-11-27 08:34:23 [DEBUG   ] Got command BAM from MCS: ref #642325
2018-11-27 08:34:23 [INFO    ] Finished processing BAM request
2018-11-27 08:34:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:34:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642325   8 58449 30863037 A  ERROR'
2018-11-27 08:34:23 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:34:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:34:23 [DEBUG   ] spawn_process: at 2018-11-27 08:34:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:34:43 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:35:03 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:35:05 [DEBUG   ] Got command BAM from MCS: ref #642328
2018-11-27 08:35:05 [INFO    ] Finished processing BAM request
2018-11-27 08:35:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:35:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642328   8 58449 30905033 A  ERROR'
2018-11-27 08:35:05 [DEBUG   ] Got command BAM from MCS: ref #642327
2018-11-27 08:35:05 [INFO    ] Finished processing BAM request
2018-11-27 08:35:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:35:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642327   8 58449 30905037 A  ERROR'
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:35:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:35:05 [DEBUG   ] spawn_process: at 2018-11-27 08:35:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:35:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 08:35:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:35:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 08:35:10 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:35:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:35:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:35:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:35:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:35:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:35:10 [DEBUG   ] Finished updating board flags in 1.041 seconds
2018-11-27 08:35:23 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 08:35:43 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:35:47 [DEBUG   ] Got command BAM from MCS: ref #642329
2018-11-27 08:35:47 [INFO    ] Finished processing BAM request
2018-11-27 08:35:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:35:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642329   8 58449 30947029 A  ERROR'
2018-11-27 08:35:47 [DEBUG   ] Got command BAM from MCS: ref #642330
2018-11-27 08:35:47 [INFO    ] Finished processing BAM request
2018-11-27 08:35:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:35:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642330   8 58449 30947033 A  ERROR'
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 08:35:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 08:35:47 [DEBUG   ] spawn_process: at 2018-11-27 08:35:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 08:35:48 [DEBUG   ] Got command RPT from MCS: ref #642331
2018-11-27 08:35:48 [DEBUG   ] summary = ERROR
2018-11-27 08:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642331  13 58449 30948977 A  ERRORERROR'
2018-11-27 08:35:48 [DEBUG   ] Got command RPT from MCS: ref #642332
2018-11-27 08:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642332  77 58449 30948978 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:36:03 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:36:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 08:36:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:36:10 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:36:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:36:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:36:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:36:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:36:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:36:10 [DEBUG   ] Finished updating board flags in 1.030 seconds
2018-11-27 08:36:23 [DEBUG   ] Finished updating ant stats in 0.399 seconds
2018-11-27 08:36:29 [DEBUG   ] Got command BAM from MCS: ref #642348
2018-11-27 08:36:29 [INFO    ] Finished processing BAM request
2018-11-27 08:36:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:36:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642348   8 58449 30989029 A  ERROR'
2018-11-27 08:36:29 [DEBUG   ] Got command BAM from MCS: ref #642349
2018-11-27 08:36:29 [INFO    ] Finished processing BAM request
2018-11-27 08:36:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:36:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642349   8 58449 30989033 A  ERROR'
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 08:36:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 08:36:29 [DEBUG   ] spawn_process: at 2018-11-27 08:36:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 08:36:44 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 08:37:04 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 08:37:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 08:37:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:37:10 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:37:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:37:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:37:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:37:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:37:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:37:10 [DEBUG   ] Finished updating board flags in 1.022 seconds
2018-11-27 08:37:12 [DEBUG   ] Got command BAM from MCS: ref #642350
2018-11-27 08:37:12 [INFO    ] Finished processing BAM request
2018-11-27 08:37:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:37:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642350   8 58449 31032025 A  ERROR'
2018-11-27 08:37:12 [DEBUG   ] Got command BAM from MCS: ref #642351
2018-11-27 08:37:12 [INFO    ] Finished processing BAM request
2018-11-27 08:37:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:37:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642351   8 58449 31032029 A  ERROR'
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 08:37:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 08:37:12 [DEBUG   ] spawn_process: at 2018-11-27 08:37:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 08:37:24 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 08:37:44 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 08:37:55 [DEBUG   ] Got command BAM from MCS: ref #642356
2018-11-27 08:37:55 [INFO    ] Finished processing BAM request
2018-11-27 08:37:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:37:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642356   8 58449 31075029 A  ERROR'
2018-11-27 08:37:55 [DEBUG   ] Got command BAM from MCS: ref #642357
2018-11-27 08:37:55 [INFO    ] Finished processing BAM request
2018-11-27 08:37:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:37:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642357   8 58449 31075033 A  ERROR'
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 08:37:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 08:37:55 [DEBUG   ] spawn_process: at 2018-11-27 08:37:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 08:38:01 [DEBUG   ] Got command RPT from MCS: ref #642358
2018-11-27 08:38:01 [DEBUG   ] summary = ERROR
2018-11-27 08:38:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642358  13 58449 31081189 A  ERRORERROR'
2018-11-27 08:38:01 [DEBUG   ] Got command RPT from MCS: ref #642359
2018-11-27 08:38:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:38:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642359  77 58449 31081381 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:38:01 [DEBUG   ] Got command RPT from MCS: ref #642360
2018-11-27 08:38:01 [DEBUG   ] summary = ERROR
2018-11-27 08:38:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642360  13 58449 31081585 A  ERRORERROR'
2018-11-27 08:38:01 [DEBUG   ] Got command RPT from MCS: ref #642361
2018-11-27 08:38:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:38:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642361  77 58449 31081789 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:38:01 [DEBUG   ] Got command RPT from MCS: ref #642362
2018-11-27 08:38:01 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:38:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642362  12 58449 31081993 A  ERRORB¡ff'
2018-11-27 08:38:04 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 08:38:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:38:10 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 08:38:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:38:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:38:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:38:10 [WARNING ] (13, 1, 'bot')
2018-11-27 08:38:10 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:38:10 [DEBUG   ] Finished updating board flags in 0.953 seconds
2018-11-27 08:38:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 08:38:38 [DEBUG   ] Got command BAM from MCS: ref #642364
2018-11-27 08:38:38 [INFO    ] Finished processing BAM request
2018-11-27 08:38:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:38:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642364   8 58449 31118033 A  ERROR'
2018-11-27 08:38:38 [DEBUG   ] Got command BAM from MCS: ref #642363
2018-11-27 08:38:38 [INFO    ] Finished processing BAM request
2018-11-27 08:38:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 08:38:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   642363   8 58449 31118037 A  ERROR'
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 08:38:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 08:38:38 [DEBUG   ] spawn_process: at 2018-11-27 08:38:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 08:38:44 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 08:39:04 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 08:39:10 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 08:39:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:39:11 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 08:39:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:39:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:39:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:39:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:39:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:39:11 [DEBUG   ] Finished updating board flags in 1.385 seconds
2018-11-27 08:39:15 [DEBUG   ] Got command STP from MCS: ref #642365
2018-11-27 08:39:15 [INFO    ] Finished processing BAM request
2018-11-27 08:39:15 [INFO    ] Shutdown the adder chain on beam #3
2018-11-27 08:39:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   642365   8 58449 31155029 A  ERROR'
2018-11-27 08:39:15 [DEBUG   ] Got command STP from MCS: ref #642366
2018-11-27 08:39:15 [INFO    ] Finished processing BAM request
2018-11-27 08:39:15 [INFO    ] Shutdown the adder chain on beam #4
2018-11-27 08:39:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   642366   8 58449 31155033 A  ERROR'
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 08:39:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 08:39:15 [DEBUG   ] spawn_process: at 2018-11-27 08:39:15 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 08:39:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:39:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:40:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:40:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:40:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:40:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:40:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:40:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:40:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:40:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:40:11 [DEBUG   ] Finished updating board flags in 1.182 seconds
2018-11-27 08:40:24 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 08:40:44 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 08:40:48 [DEBUG   ] Got command RPT from MCS: ref #642371
2018-11-27 08:40:48 [DEBUG   ] summary = ERROR
2018-11-27 08:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642371  13 58449 31248977 A  ERRORERROR'
2018-11-27 08:40:48 [DEBUG   ] Got command RPT from MCS: ref #642372
2018-11-27 08:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642372  77 58449 31248978 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:41:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:41:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:41:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:41:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:41:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:41:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:41:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:41:11 [DEBUG   ] Finished updating board flags in 1.000 seconds
2018-11-27 08:41:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:41:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:42:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:42:10 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:42:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:42:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:42:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:42:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:42:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:42:11 [DEBUG   ] Finished updating board flags in 1.061 seconds
2018-11-27 08:42:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:42:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:43:01 [DEBUG   ] Got command RPT from MCS: ref #642390
2018-11-27 08:43:01 [DEBUG   ] summary = ERROR
2018-11-27 08:43:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642390  13 58449 31381277 A  ERRORERROR'
2018-11-27 08:43:01 [DEBUG   ] Got command RPT from MCS: ref #642391
2018-11-27 08:43:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:43:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642391  77 58449 31381469 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:43:01 [DEBUG   ] Got command RPT from MCS: ref #642392
2018-11-27 08:43:01 [DEBUG   ] summary = ERROR
2018-11-27 08:43:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642392  13 58449 31381673 A  ERRORERROR'
2018-11-27 08:43:01 [DEBUG   ] Got command RPT from MCS: ref #642394
2018-11-27 08:43:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:43:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642394  77 58449 31381877 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:43:02 [DEBUG   ] Got command RPT from MCS: ref #642396
2018-11-27 08:43:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:43:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642396  12 58449 31382081 A  ERRORB™™š'
2018-11-27 08:43:04 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 08:43:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:43:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:43:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:43:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:43:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:43:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:43:12 [DEBUG   ] Finished updating board flags in 2.005 seconds
2018-11-27 08:43:24 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 08:43:44 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:44:04 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:44:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:44:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:44:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:44:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:44:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:44:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:44:11 [DEBUG   ] Finished updating board flags in 1.368 seconds
2018-11-27 08:44:24 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:44:44 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:45:04 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:45:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:45:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:45:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:45:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:45:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:45:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:45:11 [DEBUG   ] Finished updating board flags in 1.246 seconds
2018-11-27 08:45:24 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:45:45 [DEBUG   ] Finished updating ant stats in 1.070 seconds
2018-11-27 08:45:48 [DEBUG   ] Got command RPT from MCS: ref #642404
2018-11-27 08:45:48 [DEBUG   ] summary = ERROR
2018-11-27 08:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642404  13 58449 31548973 A  ERRORERROR'
2018-11-27 08:45:48 [DEBUG   ] Got command RPT from MCS: ref #642405
2018-11-27 08:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642405  77 58449 31548974 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:46:05 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 08:46:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:46:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:46:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:46:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:46:11 [WARNING ] (13, 1, 'bot')
2018-11-27 08:46:11 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:46:11 [DEBUG   ] Finished updating board flags in 0.889 seconds
2018-11-27 08:46:25 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 08:46:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:47:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:47:11 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:47:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:47:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:47:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:47:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:47:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:47:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:47:12 [DEBUG   ] Finished updating board flags in 1.370 seconds
2018-11-27 08:47:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 08:47:45 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 08:48:01 [DEBUG   ] Got command RPT from MCS: ref #642425
2018-11-27 08:48:01 [DEBUG   ] summary = ERROR
2018-11-27 08:48:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642425  13 58449 31681357 A  ERRORERROR'
2018-11-27 08:48:01 [DEBUG   ] Got command RPT from MCS: ref #642426
2018-11-27 08:48:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:48:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642426  77 58449 31681561 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:48:01 [DEBUG   ] Got command RPT from MCS: ref #642427
2018-11-27 08:48:01 [DEBUG   ] summary = ERROR
2018-11-27 08:48:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642427  13 58449 31681765 A  ERRORERROR'
2018-11-27 08:48:01 [DEBUG   ] Got command RPT from MCS: ref #642428
2018-11-27 08:48:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:48:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642428  77 58449 31681957 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:48:02 [DEBUG   ] Got command RPT from MCS: ref #642429
2018-11-27 08:48:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:48:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642429  12 58449 31682161 A  ERRORB˜™š'
2018-11-27 08:48:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 08:48:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:48:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 08:48:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:48:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:48:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:48:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:48:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:48:12 [DEBUG   ] Finished updating board flags in 1.208 seconds
2018-11-27 08:48:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:48:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:49:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:49:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:49:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:49:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:49:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:49:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:49:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:49:12 [DEBUG   ] Finished updating board flags in 1.238 seconds
2018-11-27 08:49:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:49:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:50:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:50:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:50:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:50:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:50:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:50:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:50:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:50:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:50:12 [DEBUG   ] Finished updating board flags in 1.235 seconds
2018-11-27 08:50:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:50:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:50:48 [DEBUG   ] Got command RPT from MCS: ref #642434
2018-11-27 08:50:48 [DEBUG   ] summary = ERROR
2018-11-27 08:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642434  13 58449 31848973 A  ERRORERROR'
2018-11-27 08:50:48 [DEBUG   ] Got command RPT from MCS: ref #642435
2018-11-27 08:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642435  77 58449 31848974 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:51:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 08:51:11 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:51:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:51:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:51:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:51:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:51:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:51:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:51:12 [DEBUG   ] Finished updating board flags in 1.128 seconds
2018-11-27 08:51:25 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 08:51:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:52:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:52:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:52:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:52:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:52:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:52:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:52:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:52:12 [DEBUG   ] Finished updating board flags in 1.005 seconds
2018-11-27 08:52:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:52:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:53:01 [DEBUG   ] Got command RPT from MCS: ref #642457
2018-11-27 08:53:01 [DEBUG   ] summary = ERROR
2018-11-27 08:53:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642457  13 58449 31981441 A  ERRORERROR'
2018-11-27 08:53:01 [DEBUG   ] Got command RPT from MCS: ref #642458
2018-11-27 08:53:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:53:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642458  77 58449 31981645 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:53:01 [DEBUG   ] Got command RPT from MCS: ref #642459
2018-11-27 08:53:01 [DEBUG   ] summary = ERROR
2018-11-27 08:53:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642459  13 58449 31981849 A  ERRORERROR'
2018-11-27 08:53:02 [DEBUG   ] Got command RPT from MCS: ref #642461
2018-11-27 08:53:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:53:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642461  77 58449 31982053 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:53:02 [DEBUG   ] Got command RPT from MCS: ref #642462
2018-11-27 08:53:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:53:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642462  12 58449 31982245 A  ERRORB—™š'
2018-11-27 08:53:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:53:11 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:53:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:53:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:53:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:53:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:53:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:53:12 [DEBUG   ] Finished updating board flags in 0.987 seconds
2018-11-27 08:53:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:53:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:54:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:54:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:54:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:54:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:54:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:54:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:54:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:54:12 [DEBUG   ] Finished updating board flags in 1.284 seconds
2018-11-27 08:54:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:54:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:55:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 08:55:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:55:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:55:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:55:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:55:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:55:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:55:12 [DEBUG   ] Finished updating board flags in 1.168 seconds
2018-11-27 08:55:25 [DEBUG   ] Finished updating ant stats in 0.594 seconds
2018-11-27 08:55:45 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2018-11-27 08:55:48 [DEBUG   ] Got command RPT from MCS: ref #642471
2018-11-27 08:55:48 [DEBUG   ] summary = ERROR
2018-11-27 08:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642471  13 58449 32148985 A  ERRORERROR'
2018-11-27 08:55:48 [DEBUG   ] Got command RPT from MCS: ref #642472
2018-11-27 08:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642472  77 58449 32148986 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:56:05 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 08:56:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:56:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:56:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:56:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:56:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:56:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:56:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:56:12 [DEBUG   ] Finished updating board flags in 1.027 seconds
2018-11-27 08:56:25 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 08:56:46 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 08:57:06 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 08:57:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 08:57:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:57:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 08:57:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:57:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:57:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:57:12 [WARNING ] (13, 1, 'bot')
2018-11-27 08:57:12 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:57:12 [DEBUG   ] Finished updating board flags in 0.975 seconds
2018-11-27 08:57:26 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 08:57:46 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 08:58:01 [DEBUG   ] Got command RPT from MCS: ref #642488
2018-11-27 08:58:01 [DEBUG   ] summary = ERROR
2018-11-27 08:58:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642488  13 58449 32281537 A  ERRORERROR'
2018-11-27 08:58:01 [DEBUG   ] Got command RPT from MCS: ref #642489
2018-11-27 08:58:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:58:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642489  77 58449 32281729 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:58:01 [DEBUG   ] Got command RPT from MCS: ref #642490
2018-11-27 08:58:01 [DEBUG   ] summary = ERROR
2018-11-27 08:58:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642490  13 58449 32281933 A  ERRORERROR'
2018-11-27 08:58:02 [DEBUG   ] Got command RPT from MCS: ref #642491
2018-11-27 08:58:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 08:58:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642491  77 58449 32282137 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 08:58:02 [DEBUG   ] Got command RPT from MCS: ref #642492
2018-11-27 08:58:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 08:58:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642492  12 58449 32282341 A  ERRORB˜™š'
2018-11-27 08:58:06 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 08:58:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:58:13 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 08:58:13 [WARNING ] (13, 1, 'bot')
2018-11-27 08:58:13 [WARNING ] (13, 1, 'bot')
2018-11-27 08:58:13 [WARNING ] (13, 1, 'bot')
2018-11-27 08:58:13 [WARNING ] (13, 1, 'bot')
2018-11-27 08:58:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:58:13 [DEBUG   ] Finished updating board flags in 1.426 seconds
2018-11-27 08:58:26 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 08:58:46 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 08:59:06 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 08:59:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 08:59:13 [WARNING ] (13, 1, 'bot')
2018-11-27 08:59:13 [WARNING ] (13, 1, 'bot')
2018-11-27 08:59:13 [WARNING ] (13, 1, 'bot')
2018-11-27 08:59:13 [WARNING ] (13, 1, 'bot')
2018-11-27 08:59:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 08:59:13 [DEBUG   ] Finished updating board flags in 1.418 seconds
2018-11-27 08:59:26 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 08:59:46 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 09:00:00 [DEBUG   ] Got command TBN from MCS: ref #642497
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '1', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '2', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] tbn = exited with status True, code 0x00
2018-11-27 09:00:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBN   642497   8 58449 32400905 A  ERROR'
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '3', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '4', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '5', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '6', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '7', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '8', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '9', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '10', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '11', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '12', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '13', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '14', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '15', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '16', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '17', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '18', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '19', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '20', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '21', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '22', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '23', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '24', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '25', '1543309202']
2018-11-27 09:00:00 [DEBUG   ] spawn_process: at 2018-11-27 09:00:00 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '38100000.0', '7', '20', '0', '26', '1543309202']
2018-11-27 09:00:00 [INFO    ] Finished spawning TBN in 0.050 s
2018-11-27 09:00:06 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 09:00:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:00:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:00:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:00:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:00:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:00:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:00:13 [DEBUG   ] Finished updating board flags in 1.665 seconds
2018-11-27 09:00:26 [DEBUG   ] Finished updating ant stats in 1.266 seconds
2018-11-27 09:00:46 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 09:00:48 [DEBUG   ] Got command RPT from MCS: ref #642498
2018-11-27 09:00:48 [DEBUG   ] summary = ERROR
2018-11-27 09:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642498  13 58449 32448977 A  ERRORERROR'
2018-11-27 09:00:48 [DEBUG   ] Got command RPT from MCS: ref #642499
2018-11-27 09:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642499  77 58449 32448978 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:01:06 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 09:01:12 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:01:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:01:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2018-11-27 09:01:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:01:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:01:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:01:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:01:13 [WARNING ] (25, 5, 'bot')
2018-11-27 09:01:13 [WARNING ] (25, 5, 'bot')
2018-11-27 09:01:13 [WARNING ] (25, 5, 'bot')
2018-11-27 09:01:13 [WARNING ] (25, 5, 'bot')
2018-11-27 09:01:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:01:13 [DEBUG   ] Finished updating board flags in 1.473 seconds
2018-11-27 09:01:26 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 09:01:46 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 09:02:06 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 09:02:12 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:02:12 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:02:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:02:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:02:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:02:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:02:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:02:13 [DEBUG   ] Finished updating board flags in 1.481 seconds
2018-11-27 09:02:26 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 09:02:46 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 09:03:01 [DEBUG   ] Got command RPT from MCS: ref #642521
2018-11-27 09:03:01 [DEBUG   ] summary = ERROR
2018-11-27 09:03:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642521  13 58449 32581613 A  ERRORERROR'
2018-11-27 09:03:01 [DEBUG   ] Got command RPT from MCS: ref #642522
2018-11-27 09:03:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:03:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642522  77 58449 32581817 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:03:02 [DEBUG   ] Got command RPT from MCS: ref #642523
2018-11-27 09:03:02 [DEBUG   ] summary = ERROR
2018-11-27 09:03:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642523  13 58449 32582021 A  ERRORERROR'
2018-11-27 09:03:02 [DEBUG   ] Got command RPT from MCS: ref #642525
2018-11-27 09:03:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:03:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642525  77 58449 32582225 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:03:02 [DEBUG   ] Got command RPT from MCS: ref #642527
2018-11-27 09:03:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:03:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642527  12 58449 32582417 A  ERRORB—™š'
2018-11-27 09:03:06 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 09:03:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:03:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:03:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:03:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:03:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:03:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:03:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:03:13 [DEBUG   ] Finished updating board flags in 1.494 seconds
2018-11-27 09:03:26 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:03:46 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 09:04:06 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 09:04:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:04:13 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 09:04:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:04:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:04:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:04:13 [WARNING ] (13, 1, 'bot')
2018-11-27 09:04:13 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:04:13 [DEBUG   ] Finished updating board flags in 1.501 seconds
2018-11-27 09:04:26 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 09:04:46 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:05:06 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:05:13 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:05:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:05:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:05:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:05:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:05:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:05:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:05:14 [DEBUG   ] Finished updating board flags in 2.154 seconds
2018-11-27 09:05:26 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:05:46 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 09:05:48 [DEBUG   ] Got command RPT from MCS: ref #642535
2018-11-27 09:05:48 [DEBUG   ] summary = ERROR
2018-11-27 09:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642535  13 58449 32748977 A  ERRORERROR'
2018-11-27 09:05:48 [DEBUG   ] Got command RPT from MCS: ref #642536
2018-11-27 09:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642536  77 58449 32748978 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:06:06 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 09:06:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:06:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 09:06:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:06:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:06:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:06:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:06:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:06:14 [DEBUG   ] Finished updating board flags in 1.968 seconds
2018-11-27 09:06:27 [DEBUG   ] Finished updating ant stats in 1.392 seconds
2018-11-27 09:06:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 09:07:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 09:07:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:07:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:07:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 09:07:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:07:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:07:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:07:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:07:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:07:14 [DEBUG   ] Finished updating board flags in 1.826 seconds
2018-11-27 09:07:27 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 09:07:47 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 09:08:01 [DEBUG   ] Got command RPT from MCS: ref #642556
2018-11-27 09:08:01 [DEBUG   ] summary = ERROR
2018-11-27 09:08:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642556  13 58449 32881701 A  ERRORERROR'
2018-11-27 09:08:01 [DEBUG   ] Got command RPT from MCS: ref #642557
2018-11-27 09:08:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:08:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642557  77 58449 32881905 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:08:02 [DEBUG   ] Got command RPT from MCS: ref #642558
2018-11-27 09:08:02 [DEBUG   ] summary = ERROR
2018-11-27 09:08:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642558  13 58449 32882109 A  ERRORERROR'
2018-11-27 09:08:02 [DEBUG   ] Got command RPT from MCS: ref #642559
2018-11-27 09:08:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:08:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642559  77 58449 32882313 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:08:02 [DEBUG   ] Got command RPT from MCS: ref #642560
2018-11-27 09:08:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:08:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642560  12 58449 32882505 A  ERRORB˜™š'
2018-11-27 09:08:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 09:08:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:08:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:08:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:08:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:08:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:08:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:08:14 [DEBUG   ] Finished updating board flags in 1.782 seconds
2018-11-27 09:08:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 09:08:47 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 09:09:07 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 09:09:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:09:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:09:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:09:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:09:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:09:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:09:14 [DEBUG   ] Finished updating board flags in 1.615 seconds
2018-11-27 09:09:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 09:09:47 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 09:10:07 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 09:10:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:10:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:10:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:10:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:10:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:10:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:10:14 [DEBUG   ] Finished updating board flags in 1.609 seconds
2018-11-27 09:10:27 [DEBUG   ] Finished updating ant stats in 0.991 seconds
2018-11-27 09:10:47 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:10:48 [DEBUG   ] Got command RPT from MCS: ref #642565
2018-11-27 09:10:48 [DEBUG   ] summary = ERROR
2018-11-27 09:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642565  13 58449 33048981 A  ERRORERROR'
2018-11-27 09:10:48 [DEBUG   ] Got command RPT from MCS: ref #642566
2018-11-27 09:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642566  77 58449 33048982 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:11:07 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:11:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:11:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:11:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:11:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:11:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:11:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:11:14 [DEBUG   ] Finished updating board flags in 1.579 seconds
2018-11-27 09:11:27 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 09:11:47 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 09:12:07 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:12:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:12:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:12:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:12:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:12:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:12:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:12:14 [DEBUG   ] Finished updating board flags in 1.420 seconds
2018-11-27 09:12:27 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:12:47 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:13:01 [DEBUG   ] Got command RPT from MCS: ref #642584
2018-11-27 09:13:01 [DEBUG   ] summary = ERROR
2018-11-27 09:13:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642584  13 58449 33181789 A  ERRORERROR'
2018-11-27 09:13:01 [DEBUG   ] Got command RPT from MCS: ref #642585
2018-11-27 09:13:01 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:13:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642585  77 58449 33181993 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:13:02 [DEBUG   ] Got command RPT from MCS: ref #642586
2018-11-27 09:13:02 [DEBUG   ] summary = ERROR
2018-11-27 09:13:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642586  13 58449 33182197 A  ERRORERROR'
2018-11-27 09:13:02 [DEBUG   ] Got command RPT from MCS: ref #642588
2018-11-27 09:13:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:13:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642588  77 58449 33182389 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:13:02 [DEBUG   ] Got command RPT from MCS: ref #642590
2018-11-27 09:13:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:13:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642590  12 58449 33182593 A  ERRORB—™š'
2018-11-27 09:13:07 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 09:13:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:13:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:13:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:13:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:13:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:13:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:13:14 [DEBUG   ] Finished updating board flags in 1.446 seconds
2018-11-27 09:13:27 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 09:13:48 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 09:14:08 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 09:14:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:14:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:14:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:14:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:14:14 [WARNING ] (13, 1, 'bot')
2018-11-27 09:14:14 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:14:14 [DEBUG   ] Finished updating board flags in 1.361 seconds
2018-11-27 09:14:28 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 09:14:48 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 09:15:08 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 09:15:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:15:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:15:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:15:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:15:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:15:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:15:15 [DEBUG   ] Finished updating board flags in 2.410 seconds
2018-11-27 09:15:28 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 09:15:48 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 09:15:48 [DEBUG   ] Got command RPT from MCS: ref #642598
2018-11-27 09:15:48 [DEBUG   ] summary = ERROR
2018-11-27 09:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642598  13 58449 33348985 A  ERRORERROR'
2018-11-27 09:15:48 [DEBUG   ] Got command RPT from MCS: ref #642599
2018-11-27 09:15:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:15:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642599  77 58449 33348986 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:16:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 09:16:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:16:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:16:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:16:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:16:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:16:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:16:15 [DEBUG   ] Finished updating board flags in 2.179 seconds
2018-11-27 09:16:28 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 09:16:48 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 09:17:08 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:17:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:17:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:17:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 09:17:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:17:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:17:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:17:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:17:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:17:15 [DEBUG   ] Finished updating board flags in 2.013 seconds
2018-11-27 09:17:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 09:17:48 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 09:18:01 [DEBUG   ] Got command RPT from MCS: ref #642619
2018-11-27 09:18:01 [DEBUG   ] summary = ERROR
2018-11-27 09:18:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642619  13 58449 33481881 A  ERRORERROR'
2018-11-27 09:18:02 [DEBUG   ] Got command RPT from MCS: ref #642620
2018-11-27 09:18:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:18:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642620  77 58449 33482085 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:18:02 [DEBUG   ] Got command RPT from MCS: ref #642621
2018-11-27 09:18:02 [DEBUG   ] summary = ERROR
2018-11-27 09:18:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642621  13 58449 33482277 A  ERRORERROR'
2018-11-27 09:18:02 [DEBUG   ] Got command RPT from MCS: ref #642622
2018-11-27 09:18:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:18:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642622  77 58449 33482481 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:18:02 [DEBUG   ] Got command RPT from MCS: ref #642623
2018-11-27 09:18:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:18:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642623  12 58449 33482685 A  ERRORB˜™š'
2018-11-27 09:18:08 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 09:18:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:18:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:18:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:18:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:18:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:18:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:18:15 [DEBUG   ] Finished updating board flags in 2.036 seconds
2018-11-27 09:18:28 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:18:48 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:19:08 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:19:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:19:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:19:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:19:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:19:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:19:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:19:15 [DEBUG   ] Finished updating board flags in 2.018 seconds
2018-11-27 09:19:28 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:19:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:20:08 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:20:14 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:20:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:20:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:20:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:20:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:20:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:20:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:20:15 [DEBUG   ] Finished updating board flags in 1.961 seconds
2018-11-27 09:20:28 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:20:48 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 09:20:48 [DEBUG   ] Got command RPT from MCS: ref #642628
2018-11-27 09:20:48 [DEBUG   ] summary = ERROR
2018-11-27 09:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642628  13 58449 33648981 A  ERRORERROR'
2018-11-27 09:20:48 [DEBUG   ] Got command RPT from MCS: ref #642629
2018-11-27 09:20:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:20:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642629  77 58449 33648982 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:21:09 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 09:21:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:21:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:21:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:21:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:21:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:21:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:21:15 [DEBUG   ] Finished updating board flags in 1.816 seconds
2018-11-27 09:21:29 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-27 09:21:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 09:22:09 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 09:22:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:22:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:22:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:22:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:22:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:22:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:22:15 [DEBUG   ] Finished updating board flags in 1.762 seconds
2018-11-27 09:22:29 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:22:49 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 09:23:01 [DEBUG   ] Got command RPT from MCS: ref #642651
2018-11-27 09:23:01 [DEBUG   ] summary = ERROR
2018-11-27 09:23:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642651  13 58449 33781961 A  ERRORERROR'
2018-11-27 09:23:02 [DEBUG   ] Got command RPT from MCS: ref #642652
2018-11-27 09:23:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:23:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642652  77 58449 33782165 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:23:02 [DEBUG   ] Got command RPT from MCS: ref #642653
2018-11-27 09:23:02 [DEBUG   ] summary = ERROR
2018-11-27 09:23:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642653  13 58449 33782369 A  ERRORERROR'
2018-11-27 09:23:02 [DEBUG   ] Got command RPT from MCS: ref #642655
2018-11-27 09:23:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:23:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642655  77 58449 33782573 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:23:02 [DEBUG   ] Got command RPT from MCS: ref #642656
2018-11-27 09:23:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:23:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642656  12 58449 33782765 A  ERRORB˜™š'
2018-11-27 09:23:09 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 09:23:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:23:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:23:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:23:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:23:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:23:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:23:15 [DEBUG   ] Finished updating board flags in 1.621 seconds
2018-11-27 09:23:29 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:23:49 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:24:09 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:24:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:24:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:24:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:24:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:24:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:24:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:24:15 [DEBUG   ] Finished updating board flags in 1.592 seconds
2018-11-27 09:24:29 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 09:24:49 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 09:25:10 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 09:25:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:25:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:25:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:25:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:25:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:25:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:25:15 [DEBUG   ] Finished updating board flags in 1.406 seconds
2018-11-27 09:25:29 [DEBUG   ] Finished updating ant stats in 0.404 seconds
2018-11-27 09:25:48 [DEBUG   ] Got command RPT from MCS: ref #642665
2018-11-27 09:25:48 [DEBUG   ] summary = ERROR
2018-11-27 09:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642665  13 58449 33948989 A  ERRORERROR'
2018-11-27 09:25:48 [DEBUG   ] Got command RPT from MCS: ref #642666
2018-11-27 09:25:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:25:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642666  77 58449 33948990 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:25:50 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2018-11-27 09:26:10 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-27 09:26:14 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:26:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:26:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:26:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:26:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:26:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:26:15 [DEBUG   ] Finished updating board flags in 1.397 seconds
2018-11-27 09:26:30 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 09:26:50 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2018-11-27 09:27:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:27:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:27:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:27:15 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 09:27:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 09:27:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:27:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:27:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:27:15 [WARNING ] (13, 1, 'bot')
2018-11-27 09:27:15 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:27:15 [DEBUG   ] Finished updating board flags in 1.352 seconds
2018-11-27 09:27:30 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 09:27:50 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 09:28:02 [DEBUG   ] Got command RPT from MCS: ref #642682
2018-11-27 09:28:02 [DEBUG   ] summary = ERROR
2018-11-27 09:28:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642682  13 58449 34082057 A  ERRORERROR'
2018-11-27 09:28:02 [DEBUG   ] Got command RPT from MCS: ref #642683
2018-11-27 09:28:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:28:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642683  77 58449 34082249 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:28:02 [DEBUG   ] Got command RPT from MCS: ref #642684
2018-11-27 09:28:02 [DEBUG   ] summary = ERROR
2018-11-27 09:28:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642684  13 58449 34082453 A  ERRORERROR'
2018-11-27 09:28:02 [DEBUG   ] Got command RPT from MCS: ref #642685
2018-11-27 09:28:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:28:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642685  77 58449 34082657 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:28:02 [DEBUG   ] Got command RPT from MCS: ref #642686
2018-11-27 09:28:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:28:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642686  12 58449 34082861 A  ERRORB˜™š'
2018-11-27 09:28:10 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 09:28:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:28:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 09:28:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:28:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:28:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:28:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:28:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:28:16 [DEBUG   ] Finished updating board flags in 2.069 seconds
2018-11-27 09:28:30 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:28:50 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:29:10 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:29:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:29:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:29:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:29:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:29:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:29:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:29:16 [DEBUG   ] Finished updating board flags in 1.981 seconds
2018-11-27 09:29:30 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:29:50 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:30:10 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:30:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:30:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:30:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:30:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:30:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:30:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:30:16 [DEBUG   ] Finished updating board flags in 1.868 seconds
2018-11-27 09:30:30 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:30:48 [DEBUG   ] Got command RPT from MCS: ref #642691
2018-11-27 09:30:48 [DEBUG   ] summary = ERROR
2018-11-27 09:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642691  13 58449 34248981 A  ERRORERROR'
2018-11-27 09:30:48 [DEBUG   ] Got command RPT from MCS: ref #642692
2018-11-27 09:30:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:30:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642692  77 58449 34248982 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:30:50 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 09:31:10 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 09:31:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:31:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:31:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:31:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:31:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:31:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:31:16 [DEBUG   ] Finished updating board flags in 1.732 seconds
2018-11-27 09:31:31 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 09:31:51 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 09:32:11 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 09:32:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:32:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:32:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:32:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:32:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:32:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:32:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:32:16 [DEBUG   ] Finished updating board flags in 1.620 seconds
2018-11-27 09:32:31 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 09:32:51 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 09:33:02 [DEBUG   ] Got command RPT from MCS: ref #642714
2018-11-27 09:33:02 [DEBUG   ] summary = ERROR
2018-11-27 09:33:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642714  13 58449 34382141 A  ERRORERROR'
2018-11-27 09:33:02 [DEBUG   ] Got command RPT from MCS: ref #642715
2018-11-27 09:33:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:33:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642715  77 58449 34382345 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:33:02 [DEBUG   ] Got command RPT from MCS: ref #642716
2018-11-27 09:33:02 [DEBUG   ] summary = ERROR
2018-11-27 09:33:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642716  13 58449 34382537 A  ERRORERROR'
2018-11-27 09:33:02 [DEBUG   ] Got command RPT from MCS: ref #642718
2018-11-27 09:33:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:33:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642718  77 58449 34382741 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:33:02 [DEBUG   ] Got command RPT from MCS: ref #642719
2018-11-27 09:33:02 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:33:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642719  12 58449 34382945 A  ERRORB˜™š'
2018-11-27 09:33:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 09:33:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:33:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:33:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:33:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:33:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:33:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:33:16 [DEBUG   ] Finished updating board flags in 1.602 seconds
2018-11-27 09:33:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 09:33:51 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 09:34:11 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 09:34:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:34:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:34:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:34:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:34:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:34:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:34:16 [DEBUG   ] Finished updating board flags in 1.606 seconds
2018-11-27 09:34:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 09:34:51 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:35:11 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 09:35:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:35:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:35:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:35:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:35:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:35:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:35:16 [DEBUG   ] Finished updating board flags in 1.595 seconds
2018-11-27 09:35:31 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 09:35:48 [DEBUG   ] Got command RPT from MCS: ref #642728
2018-11-27 09:35:48 [DEBUG   ] summary = ERROR
2018-11-27 09:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642728  13 58449 34548989 A  ERRORERROR'
2018-11-27 09:35:48 [DEBUG   ] Got command RPT from MCS: ref #642729
2018-11-27 09:35:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:35:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642729  77 58449 34548990 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:35:51 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:36:11 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:36:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:36:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:36:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:36:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:36:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:36:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:36:16 [DEBUG   ] Finished updating board flags in 1.402 seconds
2018-11-27 09:36:31 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:36:51 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 09:37:12 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 09:37:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:37:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:37:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:37:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:37:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:37:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:37:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:37:16 [DEBUG   ] Finished updating board flags in 1.403 seconds
2018-11-27 09:37:32 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 09:37:52 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 09:38:02 [DEBUG   ] Got command RPT from MCS: ref #642749
2018-11-27 09:38:02 [DEBUG   ] summary = ERROR
2018-11-27 09:38:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642749  13 58449 34682229 A  ERRORERROR'
2018-11-27 09:38:02 [DEBUG   ] Got command RPT from MCS: ref #642750
2018-11-27 09:38:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:38:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642750  77 58449 34682421 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:38:02 [DEBUG   ] Got command RPT from MCS: ref #642751
2018-11-27 09:38:02 [DEBUG   ] summary = ERROR
2018-11-27 09:38:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642751  13 58449 34682629 A  ERRORERROR'
2018-11-27 09:38:02 [DEBUG   ] Got command RPT from MCS: ref #642752
2018-11-27 09:38:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:38:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642752  77 58449 34682829 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:38:03 [DEBUG   ] Got command RPT from MCS: ref #642753
2018-11-27 09:38:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:38:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642753  12 58449 34683033 A  ERRORB—™š'
2018-11-27 09:38:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 09:38:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:38:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:38:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:38:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:38:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:38:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:38:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:38:16 [DEBUG   ] Finished updating board flags in 1.399 seconds
2018-11-27 09:38:32 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 09:38:52 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 09:39:12 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 09:39:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:39:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:39:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:39:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:39:16 [WARNING ] (13, 1, 'bot')
2018-11-27 09:39:16 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:39:16 [DEBUG   ] Finished updating board flags in 1.356 seconds
2018-11-27 09:39:32 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:39:52 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:40:12 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 09:40:16 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:40:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:40:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:40:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:40:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:40:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:40:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:40:17 [DEBUG   ] Finished updating board flags in 2.065 seconds
2018-11-27 09:40:32 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 09:40:48 [DEBUG   ] Got command RPT from MCS: ref #642758
2018-11-27 09:40:48 [DEBUG   ] summary = ERROR
2018-11-27 09:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642758  13 58449 34848996 A  ERRORERROR'
2018-11-27 09:40:48 [DEBUG   ] Got command RPT from MCS: ref #642759
2018-11-27 09:40:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:40:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642759  77 58449 34848996 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:40:52 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:41:12 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:41:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:41:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 09:41:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:41:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:41:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:41:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:41:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:41:17 [DEBUG   ] Finished updating board flags in 1.994 seconds
2018-11-27 09:41:32 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:41:52 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:42:12 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:42:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:42:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:42:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:42:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:42:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:42:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:42:17 [DEBUG   ] Finished updating board flags in 1.856 seconds
2018-11-27 09:42:32 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:42:52 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2018-11-27 09:43:02 [DEBUG   ] Got command RPT from MCS: ref #642777
2018-11-27 09:43:02 [DEBUG   ] summary = ERROR
2018-11-27 09:43:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642777  13 58449 34982313 A  ERRORERROR'
2018-11-27 09:43:02 [DEBUG   ] Got command RPT from MCS: ref #642778
2018-11-27 09:43:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:43:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642778  77 58449 34982517 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:43:02 [DEBUG   ] Got command RPT from MCS: ref #642779
2018-11-27 09:43:02 [DEBUG   ] summary = ERROR
2018-11-27 09:43:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642779  13 58449 34982709 A  ERRORERROR'
2018-11-27 09:43:02 [DEBUG   ] Got command RPT from MCS: ref #642781
2018-11-27 09:43:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:43:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642781  77 58449 34982913 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:43:03 [DEBUG   ] Got command RPT from MCS: ref #642782
2018-11-27 09:43:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:43:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642782  12 58449 34983117 A  ERRORB˜™š'
2018-11-27 09:43:12 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 09:43:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:43:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 09:43:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:43:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:43:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:43:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:43:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:43:17 [DEBUG   ] Finished updating board flags in 1.731 seconds
2018-11-27 09:43:32 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 09:43:53 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 09:44:13 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2018-11-27 09:44:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:44:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:44:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:44:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:44:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:44:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:44:17 [DEBUG   ] Finished updating board flags in 1.617 seconds
2018-11-27 09:44:33 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 09:44:53 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 09:45:13 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 09:45:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:45:17 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 09:45:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:45:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:45:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:45:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:45:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:45:17 [DEBUG   ] Finished updating board flags in 1.648 seconds
2018-11-27 09:45:33 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 09:45:48 [DEBUG   ] Got command RPT from MCS: ref #642791
2018-11-27 09:45:48 [DEBUG   ] summary = ERROR
2018-11-27 09:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642791  13 58449 35148985 A  ERRORERROR'
2018-11-27 09:45:48 [DEBUG   ] Got command RPT from MCS: ref #642792
2018-11-27 09:45:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:45:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642792  77 58449 35148986 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:45:53 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 09:46:13 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 09:46:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:46:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:46:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:46:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:46:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:46:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:46:17 [DEBUG   ] Finished updating board flags in 1.556 seconds
2018-11-27 09:46:33 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 09:46:53 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 09:47:13 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 09:47:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:47:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:47:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:47:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:47:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:47:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:47:17 [DEBUG   ] Finished updating board flags in 1.437 seconds
2018-11-27 09:47:33 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 09:47:53 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 09:48:02 [DEBUG   ] Got command RPT from MCS: ref #642812
2018-11-27 09:48:02 [DEBUG   ] summary = ERROR
2018-11-27 09:48:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642812  13 58449 35282397 A  ERRORERROR'
2018-11-27 09:48:02 [DEBUG   ] Got command RPT from MCS: ref #642813
2018-11-27 09:48:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:48:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642813  77 58449 35282601 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:48:02 [DEBUG   ] Got command RPT from MCS: ref #642814
2018-11-27 09:48:02 [DEBUG   ] summary = ERROR
2018-11-27 09:48:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642814  13 58449 35282805 A  ERRORERROR'
2018-11-27 09:48:02 [DEBUG   ] Got command RPT from MCS: ref #642815
2018-11-27 09:48:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:48:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642815  77 58449 35282997 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:48:03 [DEBUG   ] Got command RPT from MCS: ref #642816
2018-11-27 09:48:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:48:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642816  12 58449 35283201 A  ERRORB—™š'
2018-11-27 09:48:13 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 09:48:16 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:48:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:48:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:48:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:48:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:48:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:48:17 [DEBUG   ] Finished updating board flags in 1.454 seconds
2018-11-27 09:48:33 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:48:53 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 09:49:13 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 09:49:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:49:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:49:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:49:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:49:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:49:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:49:17 [DEBUG   ] Finished updating board flags in 1.454 seconds
2018-11-27 09:49:33 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 09:49:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 09:50:13 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 09:50:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:50:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:50:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:50:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:50:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:50:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:50:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:50:17 [DEBUG   ] Finished updating board flags in 1.455 seconds
2018-11-27 09:50:33 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:50:48 [DEBUG   ] Got command RPT from MCS: ref #642821
2018-11-27 09:50:48 [DEBUG   ] summary = ERROR
2018-11-27 09:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642821  13 58449 35448997 A  ERRORERROR'
2018-11-27 09:50:48 [DEBUG   ] Got command RPT from MCS: ref #642822
2018-11-27 09:50:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:50:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642822  77 58449 35448998 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:50:53 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:51:13 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 09:51:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:51:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:51:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:51:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:51:17 [WARNING ] (13, 1, 'bot')
2018-11-27 09:51:17 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:51:17 [DEBUG   ] Finished updating board flags in 1.470 seconds
2018-11-27 09:51:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:51:53 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:52:13 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:52:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:52:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:52:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:52:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:52:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:52:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:52:18 [DEBUG   ] Finished updating board flags in 2.208 seconds
2018-11-27 09:52:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 09:52:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:53:02 [DEBUG   ] Got command RPT from MCS: ref #642844
2018-11-27 09:53:02 [DEBUG   ] summary = ERROR
2018-11-27 09:53:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642844  13 58449 35582481 A  ERRORERROR'
2018-11-27 09:53:02 [DEBUG   ] Got command RPT from MCS: ref #642845
2018-11-27 09:53:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:53:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642845  77 58449 35582685 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:53:02 [DEBUG   ] Got command RPT from MCS: ref #642846
2018-11-27 09:53:02 [DEBUG   ] summary = ERROR
2018-11-27 09:53:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642846  13 58449 35582889 A  ERRORERROR'
2018-11-27 09:53:03 [DEBUG   ] Got command RPT from MCS: ref #642848
2018-11-27 09:53:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:53:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642848  77 58449 35583081 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:53:03 [DEBUG   ] Got command RPT from MCS: ref #642850
2018-11-27 09:53:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:53:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642850  12 58449 35583285 A  ERRORB˜™š'
2018-11-27 09:53:13 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:53:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:53:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:53:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:53:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:53:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:53:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:53:18 [DEBUG   ] Finished updating board flags in 2.150 seconds
2018-11-27 09:53:33 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:53:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:54:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:54:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:54:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:54:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:54:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:54:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:54:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:54:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:54:18 [DEBUG   ] Finished updating board flags in 2.047 seconds
2018-11-27 09:54:33 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:54:53 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:55:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:55:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:55:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:55:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:55:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:55:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:55:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:55:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:55:18 [DEBUG   ] Finished updating board flags in 1.942 seconds
2018-11-27 09:55:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:55:48 [DEBUG   ] Got command RPT from MCS: ref #642858
2018-11-27 09:55:48 [DEBUG   ] summary = ERROR
2018-11-27 09:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642858  13 58449 35748989 A  ERRORERROR'
2018-11-27 09:55:48 [DEBUG   ] Got command RPT from MCS: ref #642859
2018-11-27 09:55:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:55:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642859  77 58449 35748990 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:55:53 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 09:56:13 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 09:56:17 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:56:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:56:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:56:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:56:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:56:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:56:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:56:18 [DEBUG   ] Finished updating board flags in 1.829 seconds
2018-11-27 09:56:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:56:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 09:57:13 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2018-11-27 09:57:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 09:57:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:57:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:57:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:57:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:57:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:57:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:57:18 [DEBUG   ] Finished updating board flags in 1.851 seconds
2018-11-27 09:57:33 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 09:57:54 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 09:58:02 [DEBUG   ] Got command RPT from MCS: ref #642875
2018-11-27 09:58:02 [DEBUG   ] summary = ERROR
2018-11-27 09:58:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642875  13 58449 35882569 A  ERRORERROR'
2018-11-27 09:58:02 [DEBUG   ] Got command RPT from MCS: ref #642876
2018-11-27 09:58:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:58:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642876  77 58449 35882773 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:58:02 [DEBUG   ] Got command RPT from MCS: ref #642877
2018-11-27 09:58:02 [DEBUG   ] summary = ERROR
2018-11-27 09:58:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642877  13 58449 35882965 A  ERRORERROR'
2018-11-27 09:58:03 [DEBUG   ] Got command RPT from MCS: ref #642878
2018-11-27 09:58:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 09:58:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642878  77 58449 35883169 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 09:58:03 [DEBUG   ] Got command RPT from MCS: ref #642879
2018-11-27 09:58:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 09:58:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642879  12 58449 35883373 A  ERRORB—™š'
2018-11-27 09:58:14 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2018-11-27 09:58:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:58:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:58:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:58:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:58:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:58:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:58:18 [DEBUG   ] Finished updating board flags in 1.724 seconds
2018-11-27 09:58:34 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 09:58:54 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 09:59:14 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 09:59:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 09:59:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:59:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:59:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:59:18 [WARNING ] (13, 1, 'bot')
2018-11-27 09:59:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 09:59:18 [DEBUG   ] Finished updating board flags in 1.608 seconds
2018-11-27 09:59:34 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 09:59:54 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2018-11-27 10:00:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:00:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:00:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:00:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:00:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:00:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:00:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:00:18 [DEBUG   ] Finished updating board flags in 1.586 seconds
2018-11-27 10:00:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 10:00:48 [DEBUG   ] Got command RPT from MCS: ref #642884
2018-11-27 10:00:48 [DEBUG   ] summary = ERROR
2018-11-27 10:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642884  13 58449 36048997 A  ERRORERROR'
2018-11-27 10:00:48 [DEBUG   ] Got command RPT from MCS: ref #642885
2018-11-27 10:00:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:00:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642885  77 58449 36048998 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:00:54 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 10:01:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:01:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:01:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:01:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:01:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:01:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:01:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:01:18 [DEBUG   ] Finished updating board flags in 1.438 seconds
2018-11-27 10:01:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:01:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 10:02:14 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 10:02:17 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:02:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:02:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:02:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:02:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:02:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:02:18 [DEBUG   ] Finished updating board flags in 1.379 seconds
2018-11-27 10:02:34 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 10:02:54 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:03:02 [DEBUG   ] Got command RPT from MCS: ref #642907
2018-11-27 10:03:02 [DEBUG   ] summary = ERROR
2018-11-27 10:03:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642907  13 58449 36182653 A  ERRORERROR'
2018-11-27 10:03:02 [DEBUG   ] Got command RPT from MCS: ref #642908
2018-11-27 10:03:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:03:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642908  77 58449 36182857 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:03:03 [DEBUG   ] Got command RPT from MCS: ref #642909
2018-11-27 10:03:03 [DEBUG   ] summary = ERROR
2018-11-27 10:03:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642909  13 58449 36183061 A  ERRORERROR'
2018-11-27 10:03:03 [DEBUG   ] Got command RPT from MCS: ref #642911
2018-11-27 10:03:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:03:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642911  77 58449 36183253 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:03:03 [DEBUG   ] Got command RPT from MCS: ref #642913
2018-11-27 10:03:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:03:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642913  12 58449 36183457 A  ERRORB˜™š'
2018-11-27 10:03:14 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 10:03:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:03:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:03:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:03:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:03:18 [WARNING ] (13, 1, 'bot')
2018-11-27 10:03:18 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:03:18 [DEBUG   ] Finished updating board flags in 1.410 seconds
2018-11-27 10:03:34 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2018-11-27 10:03:54 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2018-11-27 10:04:15 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 10:04:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:04:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:04:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:04:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:04:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:04:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:04:19 [DEBUG   ] Finished updating board flags in 2.253 seconds
2018-11-27 10:04:35 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-27 10:04:55 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 10:05:15 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 10:05:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:05:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:05:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:05:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:05:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:05:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:05:19 [DEBUG   ] Finished updating board flags in 2.145 seconds
2018-11-27 10:05:35 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 10:05:48 [DEBUG   ] Got command RPT from MCS: ref #642921
2018-11-27 10:05:48 [DEBUG   ] summary = ERROR
2018-11-27 10:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642921  13 58449 36348993 A  ERRORERROR'
2018-11-27 10:05:48 [DEBUG   ] Got command RPT from MCS: ref #642922
2018-11-27 10:05:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:05:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642922  77 58449 36348994 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:05:55 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:06:15 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:06:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:06:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:06:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:06:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:06:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:06:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:06:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:06:19 [DEBUG   ] Finished updating board flags in 2.012 seconds
2018-11-27 10:06:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 10:06:55 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 10:07:15 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:07:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:07:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:07:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:07:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:07:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:07:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:07:19 [DEBUG   ] Finished updating board flags in 2.030 seconds
2018-11-27 10:07:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 10:07:55 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:08:02 [DEBUG   ] Got command RPT from MCS: ref #642942
2018-11-27 10:08:02 [DEBUG   ] summary = ERROR
2018-11-27 10:08:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642942  13 58449 36482737 A  ERRORERROR'
2018-11-27 10:08:02 [DEBUG   ] Got command RPT from MCS: ref #642943
2018-11-27 10:08:02 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:08:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642943  77 58449 36482941 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:08:03 [DEBUG   ] Got command RPT from MCS: ref #642944
2018-11-27 10:08:03 [DEBUG   ] summary = ERROR
2018-11-27 10:08:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642944  13 58449 36483145 A  ERRORERROR'
2018-11-27 10:08:03 [DEBUG   ] Got command RPT from MCS: ref #642945
2018-11-27 10:08:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:08:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642945  77 58449 36483349 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:08:03 [DEBUG   ] Got command RPT from MCS: ref #642946
2018-11-27 10:08:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:08:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642946  12 58449 36483541 A  ERRORB˜™š'
2018-11-27 10:08:15 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 10:08:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:08:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:08:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:08:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:08:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:08:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:08:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:08:19 [DEBUG   ] Finished updating board flags in 2.025 seconds
2018-11-27 10:08:35 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 10:08:55 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 10:09:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:09:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:09:19 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:09:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:09:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:09:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:09:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:09:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:09:19 [DEBUG   ] Finished updating board flags in 1.938 seconds
2018-11-27 10:09:35 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 10:09:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:10:15 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 10:10:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:10:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:10:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:10:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:10:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:10:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:10:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:10:19 [DEBUG   ] Finished updating board flags in 1.828 seconds
2018-11-27 10:10:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:10:48 [DEBUG   ] Got command RPT from MCS: ref #642951
2018-11-27 10:10:48 [DEBUG   ] summary = ERROR
2018-11-27 10:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642951  13 58449 36648997 A  ERRORERROR'
2018-11-27 10:10:48 [DEBUG   ] Got command RPT from MCS: ref #642952
2018-11-27 10:10:48 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:10:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642952  77 58449 36648998 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:10:55 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 10:11:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:11:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:11:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:11:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:11:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:11:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:11:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:11:19 [DEBUG   ] Finished updating board flags in 1.807 seconds
2018-11-27 10:11:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 10:11:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:12:15 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 10:12:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:12:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:12:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:12:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:12:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:12:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:12:19 [DEBUG   ] Finished updating board flags in 1.794 seconds
2018-11-27 10:12:35 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 10:12:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 10:13:02 [DEBUG   ] Got command RPT from MCS: ref #642970
2018-11-27 10:13:02 [DEBUG   ] summary = ERROR
2018-11-27 10:13:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642970  13 58449 36782825 A  ERRORERROR'
2018-11-27 10:13:03 [DEBUG   ] Got command RPT from MCS: ref #642971
2018-11-27 10:13:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:13:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642971  77 58449 36783029 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:13:03 [DEBUG   ] Got command RPT from MCS: ref #642972
2018-11-27 10:13:03 [DEBUG   ] summary = ERROR
2018-11-27 10:13:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642972  13 58449 36783233 A  ERRORERROR'
2018-11-27 10:13:03 [DEBUG   ] Got command RPT from MCS: ref #642974
2018-11-27 10:13:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:13:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642974  77 58449 36783437 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:13:03 [DEBUG   ] Got command RPT from MCS: ref #642975
2018-11-27 10:13:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:13:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642975  12 58449 36783629 A  ERRORB˜™š'
2018-11-27 10:13:16 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2018-11-27 10:13:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:13:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:13:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:13:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:13:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:13:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:13:19 [DEBUG   ] Finished updating board flags in 1.619 seconds
2018-11-27 10:13:36 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 10:13:56 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 10:14:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:14:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:14:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:14:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:14:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:14:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:14:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:14:19 [DEBUG   ] Finished updating board flags in 1.560 seconds
2018-11-27 10:14:36 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 10:14:56 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:15:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:15:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:15:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:15:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:15:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:15:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:15:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:15:19 [DEBUG   ] Finished updating board flags in 1.411 seconds
2018-11-27 10:15:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:15:49 [DEBUG   ] Got command RPT from MCS: ref #642984
2018-11-27 10:15:49 [DEBUG   ] summary = ERROR
2018-11-27 10:15:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642984  13 58449 36949005 A  ERRORERROR'
2018-11-27 10:15:49 [DEBUG   ] Got command RPT from MCS: ref #642985
2018-11-27 10:15:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:15:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   642985  77 58449 36949006 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:15:56 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 10:16:16 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:16:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:16:18 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:16:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:16:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:16:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:16:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:16:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:16:19 [DEBUG   ] Finished updating board flags in 1.554 seconds
2018-11-27 10:16:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:16:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:17:16 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 10:17:18 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:17:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:17:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:17:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:17:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:17:19 [WARNING ] (13, 1, 'bot')
2018-11-27 10:17:19 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:17:19 [DEBUG   ] Finished updating board flags in 1.559 seconds
2018-11-27 10:17:36 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:17:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:18:02 [DEBUG   ] Got command RPT from MCS: ref #643005
2018-11-27 10:18:02 [DEBUG   ] summary = ERROR
2018-11-27 10:18:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643005  13 58449 37082921 A  ERRORERROR'
2018-11-27 10:18:03 [DEBUG   ] Got command RPT from MCS: ref #643006
2018-11-27 10:18:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:18:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643006  77 58449 37083113 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:18:03 [DEBUG   ] Got command RPT from MCS: ref #643007
2018-11-27 10:18:03 [DEBUG   ] summary = ERROR
2018-11-27 10:18:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643007  13 58449 37083317 A  ERRORERROR'
2018-11-27 10:18:03 [DEBUG   ] Got command RPT from MCS: ref #643008
2018-11-27 10:18:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:18:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643008  77 58449 37083521 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:18:03 [DEBUG   ] Got command RPT from MCS: ref #643009
2018-11-27 10:18:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:18:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643009  12 58449 37083725 A  ERRORB—™š'
2018-11-27 10:18:16 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:18:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:18:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:18:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:18:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:18:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:18:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:18:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:18:20 [DEBUG   ] Finished updating board flags in 2.198 seconds
2018-11-27 10:18:36 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2018-11-27 10:18:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:19:16 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:19:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:19:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:19:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:19:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:19:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:19:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:19:20 [DEBUG   ] Finished updating board flags in 1.885 seconds
2018-11-27 10:19:36 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 10:19:56 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 10:20:16 [DEBUG   ] Finished updating ant stats in 0.794 seconds
2018-11-27 10:20:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:20:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:20:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:20:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:20:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:20:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:20:20 [DEBUG   ] Finished updating board flags in 1.831 seconds
2018-11-27 10:20:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:20:49 [DEBUG   ] Got command RPT from MCS: ref #643014
2018-11-27 10:20:49 [DEBUG   ] summary = ERROR
2018-11-27 10:20:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643014  13 58449 37249001 A  ERRORERROR'
2018-11-27 10:20:49 [DEBUG   ] Got command RPT from MCS: ref #643015
2018-11-27 10:20:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:20:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643015  77 58449 37249002 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:20:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:21:16 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 10:21:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:21:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:21:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:21:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:21:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:21:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:21:20 [DEBUG   ] Finished updating board flags in 2.019 seconds
2018-11-27 10:21:36 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 10:21:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:22:16 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 10:22:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:22:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:22:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:22:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:22:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:22:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:22:20 [DEBUG   ] Finished updating board flags in 1.790 seconds
2018-11-27 10:22:36 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 10:22:56 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 10:23:03 [DEBUG   ] Got command RPT from MCS: ref #643037
2018-11-27 10:23:03 [DEBUG   ] summary = ERROR
2018-11-27 10:23:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643037  13 58449 37383017 A  ERRORERROR'
2018-11-27 10:23:03 [DEBUG   ] Got command RPT from MCS: ref #643038
2018-11-27 10:23:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:23:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643038  77 58449 37383205 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:23:03 [DEBUG   ] Got command RPT from MCS: ref #643039
2018-11-27 10:23:03 [DEBUG   ] summary = ERROR
2018-11-27 10:23:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643039  13 58449 37383409 A  ERRORERROR'
2018-11-27 10:23:03 [DEBUG   ] Got command RPT from MCS: ref #643041
2018-11-27 10:23:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:23:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643041  77 58449 37383613 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:23:03 [DEBUG   ] Got command RPT from MCS: ref #643043
2018-11-27 10:23:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:23:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643043  12 58449 37383805 A  ERRORB˜™š'
2018-11-27 10:23:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:23:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:23:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:23:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:23:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:23:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:23:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:23:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:23:20 [DEBUG   ] Finished updating board flags in 1.592 seconds
2018-11-27 10:23:36 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 10:23:56 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 10:24:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:24:19 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:24:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:24:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 10:24:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:24:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:24:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:24:20 [WARNING ] (13, 1, 'bot')
2018-11-27 10:24:20 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:24:20 [DEBUG   ] Finished updating board flags in 1.540 seconds
2018-11-27 10:24:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:24:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:25:16 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 10:25:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:25:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:25:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:25:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:25:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:25:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:25:21 [DEBUG   ] Finished updating board flags in 1.764 seconds
2018-11-27 10:25:36 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2018-11-27 10:25:49 [DEBUG   ] Got command RPT from MCS: ref #643051
2018-11-27 10:25:49 [DEBUG   ] summary = ERROR
2018-11-27 10:25:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643051  13 58449 37549005 A  ERRORERROR'
2018-11-27 10:25:49 [DEBUG   ] Got command RPT from MCS: ref #643052
2018-11-27 10:25:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:25:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643052  77 58449 37549006 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:25:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 10:26:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 10:26:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:26:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:26:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:26:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:26:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:26:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:26:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:26:21 [DEBUG   ] Finished updating board flags in 2.194 seconds
2018-11-27 10:26:37 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-27 10:26:57 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 10:27:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:27:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:27:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:27:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:27:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:27:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:27:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:27:21 [DEBUG   ] Finished updating board flags in 1.888 seconds
2018-11-27 10:27:37 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 10:27:57 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:28:03 [DEBUG   ] Got command RPT from MCS: ref #643068
2018-11-27 10:28:03 [DEBUG   ] summary = ERROR
2018-11-27 10:28:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643068  13 58449 37683093 A  ERRORERROR'
2018-11-27 10:28:03 [DEBUG   ] Got command RPT from MCS: ref #643069
2018-11-27 10:28:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:28:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643069  77 58449 37683297 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:28:03 [DEBUG   ] Got command RPT from MCS: ref #643070
2018-11-27 10:28:03 [DEBUG   ] summary = ERROR
2018-11-27 10:28:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643070  13 58449 37683489 A  ERRORERROR'
2018-11-27 10:28:03 [DEBUG   ] Got command RPT from MCS: ref #643071
2018-11-27 10:28:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:28:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643071  77 58449 37683693 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:28:03 [DEBUG   ] Got command RPT from MCS: ref #643072
2018-11-27 10:28:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:28:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643072  12 58449 37683897 A  ERRORB—™š'
2018-11-27 10:28:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:28:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:28:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:28:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:28:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:28:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:28:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:28:21 [DEBUG   ] Finished updating board flags in 1.817 seconds
2018-11-27 10:28:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:28:57 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:29:17 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:29:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:29:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:29:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:29:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:29:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:29:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:29:21 [DEBUG   ] Finished updating board flags in 1.820 seconds
2018-11-27 10:29:37 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 10:29:57 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:30:17 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 10:30:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:30:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:30:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:30:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:30:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:30:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:30:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:30:21 [DEBUG   ] Finished updating board flags in 1.815 seconds
2018-11-27 10:30:37 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2018-11-27 10:30:49 [DEBUG   ] Got command RPT from MCS: ref #643077
2018-11-27 10:30:49 [DEBUG   ] summary = ERROR
2018-11-27 10:30:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643077  13 58449 37849005 A  ERRORERROR'
2018-11-27 10:30:49 [DEBUG   ] Got command RPT from MCS: ref #643078
2018-11-27 10:30:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:30:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643078  77 58449 37849006 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:30:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:31:17 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 10:31:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:31:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:31:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:31:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:31:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:31:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:31:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:31:21 [DEBUG   ] Finished updating board flags in 1.750 seconds
2018-11-27 10:31:37 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 10:31:57 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 10:32:17 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 10:32:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:32:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:32:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:32:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:32:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:32:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:32:21 [DEBUG   ] Finished updating board flags in 1.685 seconds
2018-11-27 10:32:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:32:57 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 10:33:03 [DEBUG   ] Got command RPT from MCS: ref #643100
2018-11-27 10:33:03 [DEBUG   ] summary = ERROR
2018-11-27 10:33:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643100  13 58449 37983173 A  ERRORERROR'
2018-11-27 10:33:03 [DEBUG   ] Got command RPT from MCS: ref #643101
2018-11-27 10:33:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:33:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643101  77 58449 37983377 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:33:03 [DEBUG   ] Got command RPT from MCS: ref #643102
2018-11-27 10:33:03 [DEBUG   ] summary = ERROR
2018-11-27 10:33:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643102  13 58449 37983581 A  ERRORERROR'
2018-11-27 10:33:03 [DEBUG   ] Got command RPT from MCS: ref #643104
2018-11-27 10:33:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:33:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643104  77 58449 37983785 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:33:03 [DEBUG   ] Got command RPT from MCS: ref #643106
2018-11-27 10:33:03 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:33:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643106  12 58449 37983977 A  ERRORB˜™š'
2018-11-27 10:33:17 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2018-11-27 10:33:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:33:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:33:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:33:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:33:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:33:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:33:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:33:21 [DEBUG   ] Finished updating board flags in 1.607 seconds
2018-11-27 10:33:37 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 10:33:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:34:17 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 10:34:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:34:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:34:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:34:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:34:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:34:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:34:21 [DEBUG   ] Finished updating board flags in 1.513 seconds
2018-11-27 10:34:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:34:57 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 10:35:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:35:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:35:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:35:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:35:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:35:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:35:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:35:21 [DEBUG   ] Finished updating board flags in 1.493 seconds
2018-11-27 10:35:37 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 10:35:49 [DEBUG   ] Got command RPT from MCS: ref #643114
2018-11-27 10:35:49 [DEBUG   ] summary = ERROR
2018-11-27 10:35:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643114  13 58449 38149010 A  ERRORERROR'
2018-11-27 10:35:49 [DEBUG   ] Got command RPT from MCS: ref #643115
2018-11-27 10:35:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:35:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643115  77 58449 38149010 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:35:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:36:17 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 10:36:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:36:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:36:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:36:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:36:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:36:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:36:21 [DEBUG   ] Finished updating board flags in 1.499 seconds
2018-11-27 10:36:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:36:57 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2018-11-27 10:37:17 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2018-11-27 10:37:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:37:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:37:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:37:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:37:21 [WARNING ] (13, 1, 'bot')
2018-11-27 10:37:21 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:37:21 [DEBUG   ] Finished updating board flags in 1.504 seconds
2018-11-27 10:37:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:37:57 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 10:38:03 [DEBUG   ] Got command RPT from MCS: ref #643135
2018-11-27 10:38:03 [DEBUG   ] summary = ERROR
2018-11-27 10:38:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643135  13 58449 38283265 A  ERRORERROR'
2018-11-27 10:38:03 [DEBUG   ] Got command RPT from MCS: ref #643136
2018-11-27 10:38:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:38:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643136  77 58449 38283469 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:38:03 [DEBUG   ] Got command RPT from MCS: ref #643137
2018-11-27 10:38:03 [DEBUG   ] summary = ERROR
2018-11-27 10:38:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643137  13 58449 38283661 A  ERRORERROR'
2018-11-27 10:38:03 [DEBUG   ] Got command RPT from MCS: ref #643138
2018-11-27 10:38:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:38:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643138  77 58449 38283865 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:38:04 [DEBUG   ] Got command RPT from MCS: ref #643139
2018-11-27 10:38:04 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:38:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643139  12 58449 38284069 A  ERRORB—™š'
2018-11-27 10:38:17 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2018-11-27 10:38:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:38:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:38:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:38:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:38:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:38:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:38:22 [DEBUG   ] Finished updating board flags in 2.176 seconds
2018-11-27 10:38:37 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 10:38:57 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 10:39:18 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2018-11-27 10:39:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:39:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 10:39:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:39:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:39:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:39:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:39:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:39:22 [DEBUG   ] Finished updating board flags in 1.938 seconds
2018-11-27 10:39:38 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2018-11-27 10:39:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:40:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:40:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:40:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:40:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:40:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:40:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:40:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:40:22 [DEBUG   ] Finished updating board flags in 1.817 seconds
2018-11-27 10:40:38 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 10:40:49 [DEBUG   ] Got command RPT from MCS: ref #643144
2018-11-27 10:40:49 [DEBUG   ] summary = ERROR
2018-11-27 10:40:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643144  13 58449 38449005 A  ERRORERROR'
2018-11-27 10:40:49 [DEBUG   ] Got command RPT from MCS: ref #643145
2018-11-27 10:40:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:40:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643145  77 58449 38449006 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:40:58 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 10:41:18 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:41:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:41:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:41:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:41:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:41:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:41:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:41:22 [DEBUG   ] Finished updating board flags in 1.819 seconds
2018-11-27 10:41:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:41:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:42:18 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:42:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:42:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:42:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:42:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:42:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:42:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:42:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:42:22 [DEBUG   ] Finished updating board flags in 1.860 seconds
2018-11-27 10:42:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 10:42:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2018-11-27 10:43:03 [DEBUG   ] Got command RPT from MCS: ref #643163
2018-11-27 10:43:03 [DEBUG   ] summary = ERROR
2018-11-27 10:43:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643163  13 58449 38583345 A  ERRORERROR'
2018-11-27 10:43:03 [DEBUG   ] Got command RPT from MCS: ref #643164
2018-11-27 10:43:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:43:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643164  77 58449 38583549 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:43:03 [DEBUG   ] Got command RPT from MCS: ref #643165
2018-11-27 10:43:03 [DEBUG   ] summary = ERROR
2018-11-27 10:43:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643165  13 58449 38583753 A  ERRORERROR'
2018-11-27 10:43:03 [DEBUG   ] Got command RPT from MCS: ref #643166
2018-11-27 10:43:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:43:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643166  77 58449 38583957 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:43:04 [DEBUG   ] Got command RPT from MCS: ref #643168
2018-11-27 10:43:04 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:43:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643168  12 58449 38584149 A  ERRORB˜™š'
2018-11-27 10:43:18 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2018-11-27 10:43:21 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:43:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:43:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:43:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:43:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:43:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:43:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:43:22 [DEBUG   ] Finished updating board flags in 1.750 seconds
2018-11-27 10:43:38 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 10:43:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:44:18 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 10:44:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:44:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:44:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:44:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:44:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:44:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:44:22 [DEBUG   ] Finished updating board flags in 1.577 seconds
2018-11-27 10:44:38 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 10:44:58 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 10:45:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:45:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:45:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:45:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:45:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:45:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:45:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:45:22 [DEBUG   ] Finished updating board flags in 1.456 seconds
2018-11-27 10:45:38 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 10:45:49 [DEBUG   ] Got command RPT from MCS: ref #643177
2018-11-27 10:45:49 [DEBUG   ] summary = ERROR
2018-11-27 10:45:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643177  13 58449 38749001 A  ERRORERROR'
2018-11-27 10:45:49 [DEBUG   ] Got command RPT from MCS: ref #643178
2018-11-27 10:45:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:45:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643178  77 58449 38749002 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:45:58 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2018-11-27 10:46:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 10:46:22 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:46:22 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:46:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:46:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:46:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:46:22 [WARNING ] (13, 1, 'bot')
2018-11-27 10:46:22 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:46:22 [DEBUG   ] Finished updating board flags in 1.494 seconds
2018-11-27 10:46:38 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 10:46:58 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 10:47:00 [DEBUG   ] Got command RPT from MCS: ref #643198
2018-11-27 10:47:00 [DEBUG   ] summary = ERROR
2018-11-27 10:47:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643198  13 58449 38820901 A  ERRORERROR'
2018-11-27 10:47:00 [DEBUG   ] Got command RPT from MCS: ref #643199
2018-11-27 10:47:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:47:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643199  77 58449 38820902 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:47:05 [DEBUG   ] Got command INI from MCS: ref #643200
2018-11-27 10:47:05 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2018-11-27 10:47:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   643200   8 58449 38825910 ABOOTING'
2018-11-27 10:47:29 [INFO    ] -------------------
2018-11-27 10:47:29 [INFO    ]  LWA board status
2018-11-27 10:47:29 [INFO    ] -------------------
2018-11-27 10:47:29 [INFO    ] DP 1: 2 lwa27
2018-11-27 10:47:29 [INFO    ] DP 1: 3 lwa24
2018-11-27 10:47:29 [INFO    ] DP 1: 4 lwa23
2018-11-27 10:47:29 [INFO    ] DP 1: 5 lwa18
2018-11-27 10:47:29 [INFO    ] DP 1: 6 lwa10
2018-11-27 10:47:29 [INFO    ] DP 1: 7 lwa32
2018-11-27 10:47:29 [INFO    ] DP 1: 8 lwa11
2018-11-27 10:47:29 [INFO    ] DP 1: 9 lwa30
2018-11-27 10:47:29 [INFO    ] DP 1: 10 lwa12
2018-11-27 10:47:29 [INFO    ] DP 1: 11 lwa8
2018-11-27 10:47:29 [INFO    ] DP 1: 12 lwa13
2018-11-27 10:47:29 [INFO    ] DP 1: 13 lwa26
2018-11-27 10:47:29 [INFO    ] DP 1: 14 lwa14
2018-11-27 10:47:29 [INFO    ] DP 1: 16 lwa22
2018-11-27 10:47:29 [INFO    ] DP 1: 17 lwa20
2018-11-27 10:47:29 [INFO    ] DP 1: 18 lwa1
2018-11-27 10:47:29 [INFO    ] DP 1: 19 lwa5
2018-11-27 10:47:29 [INFO    ] DP 1: 20 lwa25
2018-11-27 10:47:29 [INFO    ] DP 1: 21 lwa35
2018-11-27 10:47:29 [INFO    ] DP 1: 22 lwa19
2018-11-27 10:47:29 [INFO    ] DP 1: 23 lwa21
2018-11-27 10:47:29 [INFO    ] DP 1: 24 lwa6
2018-11-27 10:47:29 [INFO    ] DP 1: 25 lwa7
2018-11-27 10:47:29 [INFO    ] DP 1: 26 lwa33
2018-11-27 10:47:29 [INFO    ] DP 1: 27 lwa15
2018-11-27 10:47:29 [INFO    ] DP 1: 28 lwa28
2018-11-27 10:47:29 [INFO    ] DP 2: 1 lwa16
2018-11-27 10:47:29 [INFO    ] DP 2: 15 lwa34
2018-11-27 10:47:29 [INFO    ] Total of 28 boards
2018-11-27 10:47:29 [INFO    ] Total of 260 stands
2018-11-27 10:47:29 [INFO    ] -------------------
2018-11-27 10:47:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13949
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13950
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13951
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13952
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13953
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13954
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13955
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13956
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13957
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13958
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13959
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13960
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13961
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13962
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13963
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13964
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13965
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13966
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13967
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13968
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13969
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13970
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13971
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13972
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13973
2018-11-27 10:47:36 [DEBUG   ] 2018-11-27 10:47:36 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13974
2018-11-27 10:47:48 [DEBUG   ] 2018-11-27 10:47:48 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13949 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13950 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13951 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13952 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13953 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13954 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13955 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13956 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13957 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13958 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13959 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13960 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13961 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13962 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13963 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13964 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13965 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13966 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13967 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13968 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13969 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13970 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13971 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13972 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13973 exited with status 0
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13974 exited with status 0
2018-11-27 10:47:49 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] spawn_process: at 2018-11-27 10:47:49 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 14001
2018-11-27 10:47:49 [DEBUG   ] 2018-11-27 10:47:49 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 14002
2018-11-27 10:47:54 [DEBUG   ] 2018-11-27 10:47:54 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 14001 exited with status 0
2018-11-27 10:47:54 [DEBUG   ] 2018-11-27 10:47:54 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 14002 exited with status 0
2018-11-27 10:47:54 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2018-11-27 10:47:54 [DEBUG   ] spawn_process: at 2018-11-27 10:47:54 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2018-11-27 10:47:54 [DEBUG   ] spawn_process: at 2018-11-27 10:47:54 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2018-11-27 10:47:54 [INFO    ] Setting Beam 1 xbars
2018-11-27 10:47:54 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2018-11-27 10:47:54 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2018-11-27 10:47:54 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2018-11-27 10:47:55 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2018-11-27 10:47:55 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2018-11-27 10:47:55 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2018-11-27 10:47:55 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2018-11-27 10:47:55 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2018-11-27 10:47:55 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2018-11-27 10:47:55 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2018-11-27 10:47:55 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2018-11-27 10:47:55 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2018-11-27 10:47:55 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2018-11-27 10:47:56 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2018-11-27 10:47:56 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2018-11-27 10:47:56 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2018-11-27 10:47:56 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2018-11-27 10:47:56 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2018-11-27 10:47:56 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2018-11-27 10:47:56 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2018-11-27 10:47:56 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2018-11-27 10:47:56 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2018-11-27 10:47:57 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2018-11-27 10:47:57 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2018-11-27 10:47:57 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2018-11-27 10:47:57 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2018-11-27 10:47:57 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2018-11-27 10:47:57 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2018-11-27 10:47:57 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2018-11-27 10:47:57 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2018-11-27 10:47:57 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2018-11-27 10:47:58 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2018-11-27 10:47:58 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2018-11-27 10:47:58 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2018-11-27 10:47:58 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2018-11-27 10:47:58 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2018-11-27 10:47:58 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2018-11-27 10:47:58 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2018-11-27 10:47:58 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2018-11-27 10:47:58 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2018-11-27 10:47:59 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2018-11-27 10:47:59 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2018-11-27 10:47:59 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2018-11-27 10:47:59 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2018-11-27 10:47:59 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2018-11-27 10:47:59 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2018-11-27 10:47:59 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2018-11-27 10:47:59 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2018-11-27 10:47:59 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2018-11-27 10:48:00 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2018-11-27 10:48:00 [INFO    ] Setting Beam 2 xbars
2018-11-27 10:48:00 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2018-11-27 10:48:00 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2018-11-27 10:48:00 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2018-11-27 10:48:00 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2018-11-27 10:48:00 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2018-11-27 10:48:00 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2018-11-27 10:48:00 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2018-11-27 10:48:00 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2018-11-27 10:48:00 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2018-11-27 10:48:01 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2018-11-27 10:48:01 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2018-11-27 10:48:01 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2018-11-27 10:48:01 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2018-11-27 10:48:01 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2018-11-27 10:48:01 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2018-11-27 10:48:01 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2018-11-27 10:48:01 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2018-11-27 10:48:01 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2018-11-27 10:48:02 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2018-11-27 10:48:02 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2018-11-27 10:48:02 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2018-11-27 10:48:02 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2018-11-27 10:48:02 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2018-11-27 10:48:02 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2018-11-27 10:48:02 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2018-11-27 10:48:02 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2018-11-27 10:48:02 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2018-11-27 10:48:03 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2018-11-27 10:48:03 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2018-11-27 10:48:03 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2018-11-27 10:48:03 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2018-11-27 10:48:03 [DEBUG   ] Got command RPT from MCS: ref #643201
2018-11-27 10:48:03 [DEBUG   ] summary = BOOTING
2018-11-27 10:48:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643201  15 58449 38883434 ABOOTINGBOOTING'
2018-11-27 10:48:03 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2018-11-27 10:48:03 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2018-11-27 10:48:03 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2018-11-27 10:48:03 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2018-11-27 10:48:03 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2018-11-27 10:48:04 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2018-11-27 10:48:04 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2018-11-27 10:48:04 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2018-11-27 10:48:04 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2018-11-27 10:48:04 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2018-11-27 10:48:04 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2018-11-27 10:48:04 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2018-11-27 10:48:04 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2018-11-27 10:48:04 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2018-11-27 10:48:05 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2018-11-27 10:48:05 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2018-11-27 10:48:05 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2018-11-27 10:48:05 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2018-11-27 10:48:05 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2018-11-27 10:48:05 [INFO    ] Setting Beam 3 xbars
2018-11-27 10:48:05 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2018-11-27 10:48:05 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2018-11-27 10:48:05 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2018-11-27 10:48:05 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2018-11-27 10:48:05 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2018-11-27 10:48:06 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2018-11-27 10:48:06 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2018-11-27 10:48:06 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2018-11-27 10:48:06 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2018-11-27 10:48:06 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2018-11-27 10:48:06 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2018-11-27 10:48:06 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2018-11-27 10:48:06 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2018-11-27 10:48:06 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2018-11-27 10:48:07 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2018-11-27 10:48:07 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2018-11-27 10:48:07 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2018-11-27 10:48:07 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2018-11-27 10:48:07 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2018-11-27 10:48:07 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2018-11-27 10:48:07 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2018-11-27 10:48:07 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2018-11-27 10:48:07 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2018-11-27 10:48:08 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2018-11-27 10:48:08 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2018-11-27 10:48:08 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2018-11-27 10:48:08 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2018-11-27 10:48:08 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2018-11-27 10:48:08 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2018-11-27 10:48:08 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2018-11-27 10:48:08 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2018-11-27 10:48:08 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2018-11-27 10:48:09 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2018-11-27 10:48:09 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2018-11-27 10:48:09 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2018-11-27 10:48:09 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2018-11-27 10:48:09 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2018-11-27 10:48:09 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2018-11-27 10:48:09 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2018-11-27 10:48:09 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2018-11-27 10:48:09 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2018-11-27 10:48:09 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2018-11-27 10:48:10 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2018-11-27 10:48:10 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2018-11-27 10:48:10 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2018-11-27 10:48:10 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2018-11-27 10:48:10 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2018-11-27 10:48:10 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2018-11-27 10:48:10 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2018-11-27 10:48:10 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2018-11-27 10:48:10 [INFO    ] Setting Beam 4 xbars
2018-11-27 10:48:10 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2018-11-27 10:48:11 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 10:48:11 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 10:48:11 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 10:48:11 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 10:48:11 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2018-11-27 10:48:11 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2018-11-27 10:48:11 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2018-11-27 10:48:11 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2018-11-27 10:48:11 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2018-11-27 10:48:12 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2018-11-27 10:48:12 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2018-11-27 10:48:12 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2018-11-27 10:48:12 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2018-11-27 10:48:12 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2018-11-27 10:48:12 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2018-11-27 10:48:12 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2018-11-27 10:48:12 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2018-11-27 10:48:12 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2018-11-27 10:48:12 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2018-11-27 10:48:13 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2018-11-27 10:48:13 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2018-11-27 10:48:13 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2018-11-27 10:48:13 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2018-11-27 10:48:13 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2018-11-27 10:48:13 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2018-11-27 10:48:13 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2018-11-27 10:48:13 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2018-11-27 10:48:13 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2018-11-27 10:48:14 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2018-11-27 10:48:14 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2018-11-27 10:48:14 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2018-11-27 10:48:14 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2018-11-27 10:48:14 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2018-11-27 10:48:14 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2018-11-27 10:48:14 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2018-11-27 10:48:14 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2018-11-27 10:48:14 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2018-11-27 10:48:15 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2018-11-27 10:48:15 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2018-11-27 10:48:15 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2018-11-27 10:48:15 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2018-11-27 10:48:15 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2018-11-27 10:48:15 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2018-11-27 10:48:15 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2018-11-27 10:48:15 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2018-11-27 10:48:15 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2018-11-27 10:48:16 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2018-11-27 10:48:16 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2018-11-27 10:48:16 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2018-11-27 10:48:16 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2018-11-27 10:48:16 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2018-11-27 10:48:16 [INFO    ] All xbars set
2018-11-27 10:48:16 [INFO    ] Setting adder chain routing for all beams
2018-11-27 10:48:21 [INFO    ] Setting beam ids
2018-11-27 10:48:31 [INFO    ] Starting beam delay calibration
2018-11-27 10:48:43 [INFO    ] Beam 1 latency is 5738
2018-11-27 10:48:43 [INFO    ] Beam 2 latency is 5752
2018-11-27 10:48:43 [INFO    ] Beam 3 latency is 5736
2018-11-27 10:48:43 [INFO    ] Beam 4 latency is 5716
2018-11-27 10:48:43 [INFO    ] Stop calibration
2018-11-27 10:48:51 [INFO    ] Beam delay calibration successful
2018-11-27 10:48:51 [INFO    ] Calibration passed
2018-11-27 10:48:51 [DEBUG   ] spawn_process: at 2018-11-27 10:48:51 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2018-11-27 10:48:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2018-11-27 10:48:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2018-11-27 10:48:51 [DEBUG   ] spawn_process: at 2018-11-27 10:48:51 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2018-11-27 10:48:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2018-11-27 10:48:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2018-11-27 10:48:51 [DEBUG   ] spawn_process: at 2018-11-27 10:48:51 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2018-11-27 10:48:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2018-11-27 10:48:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2018-11-27 10:48:51 [DEBUG   ] spawn_process: at 2018-11-27 10:48:51 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2018-11-27 10:48:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2018-11-27 10:48:51 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2018-11-27 10:48:51 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 10:48:52 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 288 ms (1543315732.290 s) with PID 15582 and time 1543315733
2018-11-27 10:48:52 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 10:48:52 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 376 ms (1543315732.378 s) with PID 15585 and time 1543315733
2018-11-27 10:48:53 [DEBUG   ] Finished updating ant stats in 0.869 seconds
2018-11-27 10:48:53 [INFO    ] Started antennas statistics background thread
2018-11-27 10:48:53 [INFO    ] Started board flags background thread
2018-11-27 10:48:53 [INFO    ] Started BAM background thread
2018-11-27 10:48:53 [INFO    ] Finished the INI process (ref. #643200) in 107.593 s
2018-11-27 10:48:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:48:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:48:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:48:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:48:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:48:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:48:54 [DEBUG   ] Finished updating board flags in 1.053 seconds
2018-11-27 10:49:13 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:49:33 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:49:53 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 10:49:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:49:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:49:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:49:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:49:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:49:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:49:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:49:54 [DEBUG   ] Finished updating board flags in 1.010 seconds
2018-11-27 10:50:05 [DEBUG   ] Got command RPT from MCS: ref #643206
2018-11-27 10:50:05 [DEBUG   ] summary = ERROR
2018-11-27 10:50:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643206  13 58449 39005913 A  ERRORERROR'
2018-11-27 10:50:05 [DEBUG   ] Got command RPT from MCS: ref #643207
2018-11-27 10:50:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:50:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643207  77 58449 39005914 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:50:13 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 10:50:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:50:49 [DEBUG   ] Got command RPT from MCS: ref #643208
2018-11-27 10:50:49 [DEBUG   ] summary = ERROR
2018-11-27 10:50:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643208  13 58449 39049009 A  ERRORERROR'
2018-11-27 10:50:49 [DEBUG   ] Got command RPT from MCS: ref #643209
2018-11-27 10:50:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:50:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643209  77 58449 39049010 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:50:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:50:53 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:50:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:50:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:50:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:50:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:50:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:50:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:50:54 [DEBUG   ] Finished updating board flags in 0.986 seconds
2018-11-27 10:51:13 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 10:51:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:51:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:51:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:51:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:51:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 10:51:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:51:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:51:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:51:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:51:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:51:55 [DEBUG   ] Finished updating board flags in 1.847 seconds
2018-11-27 10:52:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:52:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:52:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:52:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:52:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:52:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:52:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:52:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:52:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:52:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:52:55 [DEBUG   ] Finished updating board flags in 1.802 seconds
2018-11-27 10:53:03 [DEBUG   ] Got command RPT from MCS: ref #643231
2018-11-27 10:53:03 [DEBUG   ] summary = ERROR
2018-11-27 10:53:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643231  13 58449 39183521 A  ERRORERROR'
2018-11-27 10:53:03 [DEBUG   ] Got command RPT from MCS: ref #643232
2018-11-27 10:53:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:53:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643232  77 58449 39183713 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:53:03 [DEBUG   ] Got command RPT from MCS: ref #643233
2018-11-27 10:53:03 [DEBUG   ] summary = ERROR
2018-11-27 10:53:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643233  13 58449 39183917 A  ERRORERROR'
2018-11-27 10:53:04 [DEBUG   ] Got command RPT from MCS: ref #643235
2018-11-27 10:53:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:53:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643235  77 58449 39184121 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:53:04 [DEBUG   ] Got command RPT from MCS: ref #643236
2018-11-27 10:53:04 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:53:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643236  12 58449 39184325 A  ERRORB˜™š'
2018-11-27 10:53:13 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 10:53:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:53:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 10:53:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:53:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:53:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:53:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:53:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:53:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:53:54 [DEBUG   ] Finished updating board flags in 1.228 seconds
2018-11-27 10:54:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:54:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:54:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:54:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:54:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:54:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:54:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:54:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:54:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:54:55 [DEBUG   ] Finished updating board flags in 1.754 seconds
2018-11-27 10:55:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:55:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:55:49 [DEBUG   ] Got command RPT from MCS: ref #643245
2018-11-27 10:55:49 [DEBUG   ] summary = ERROR
2018-11-27 10:55:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643245  13 58449 39349013 A  ERRORERROR'
2018-11-27 10:55:49 [DEBUG   ] Got command RPT from MCS: ref #643246
2018-11-27 10:55:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:55:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643246  77 58449 39349014 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:55:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:55:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:55:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:55:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:55:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:55:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:55:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:55:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:55:54 [DEBUG   ] Finished updating board flags in 1.031 seconds
2018-11-27 10:56:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:56:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:56:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:56:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:56:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:56:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:56:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:56:54 [WARNING ] (13, 1, 'bot')
2018-11-27 10:56:54 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:56:54 [DEBUG   ] Finished updating board flags in 1.055 seconds
2018-11-27 10:57:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:57:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:57:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:57:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:57:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:57:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:57:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:57:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:57:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:57:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:57:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:57:55 [DEBUG   ] Finished updating board flags in 1.558 seconds
2018-11-27 10:58:03 [DEBUG   ] Got command RPT from MCS: ref #643262
2018-11-27 10:58:03 [DEBUG   ] summary = ERROR
2018-11-27 10:58:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643262  13 58449 39483597 A  ERRORERROR'
2018-11-27 10:58:03 [DEBUG   ] Got command RPT from MCS: ref #643263
2018-11-27 10:58:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:58:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643263  77 58449 39483801 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:58:04 [DEBUG   ] Got command RPT from MCS: ref #643264
2018-11-27 10:58:04 [DEBUG   ] summary = ERROR
2018-11-27 10:58:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643264  13 58449 39484005 A  ERRORERROR'
2018-11-27 10:58:04 [DEBUG   ] Got command RPT from MCS: ref #643265
2018-11-27 10:58:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 10:58:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643265  77 58449 39484197 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 10:58:04 [DEBUG   ] Got command RPT from MCS: ref #643266
2018-11-27 10:58:04 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 10:58:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643266  12 58449 39484401 A  ERRORB—™š'
2018-11-27 10:58:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:58:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:58:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:58:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:58:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 10:58:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 10:58:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:58:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:58:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:58:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:58:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:58:55 [DEBUG   ] Finished updating board flags in 1.382 seconds
2018-11-27 10:59:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:59:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:59:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 10:59:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 10:59:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 10:59:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:59:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:59:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:59:55 [WARNING ] (13, 1, 'bot')
2018-11-27 10:59:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 10:59:55 [DEBUG   ] Finished updating board flags in 1.190 seconds
2018-11-27 11:00:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:00:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:00:49 [DEBUG   ] Got command RPT from MCS: ref #643271
2018-11-27 11:00:49 [DEBUG   ] summary = ERROR
2018-11-27 11:00:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643271  13 58449 39649005 A  ERRORERROR'
2018-11-27 11:00:49 [DEBUG   ] Got command RPT from MCS: ref #643272
2018-11-27 11:00:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:00:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643272  77 58449 39649006 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:00:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 11:00:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:00:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:00:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:00:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:00:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:00:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:00:55 [DEBUG   ] Finished updating board flags in 1.011 seconds
2018-11-27 11:01:13 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 11:01:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:01:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:01:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:01:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:01:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:01:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:01:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:01:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:01:56 [DEBUG   ] Finished updating board flags in 2.002 seconds
2018-11-27 11:02:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:02:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 11:02:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:02:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:02:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:02:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:02:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:02:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:02:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:02:56 [DEBUG   ] Finished updating board flags in 2.027 seconds
2018-11-27 11:03:03 [DEBUG   ] Got command RPT from MCS: ref #643294
2018-11-27 11:03:03 [DEBUG   ] summary = ERROR
2018-11-27 11:03:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643294  13 58449 39783681 A  ERRORERROR'
2018-11-27 11:03:03 [DEBUG   ] Got command RPT from MCS: ref #643295
2018-11-27 11:03:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:03:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643295  77 58449 39783885 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:03:04 [DEBUG   ] Got command RPT from MCS: ref #643296
2018-11-27 11:03:04 [DEBUG   ] summary = ERROR
2018-11-27 11:03:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643296  13 58449 39784089 A  ERRORERROR'
2018-11-27 11:03:04 [DEBUG   ] Got command RPT from MCS: ref #643298
2018-11-27 11:03:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:03:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643298  77 58449 39784293 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:03:04 [DEBUG   ] Got command RPT from MCS: ref #643299
2018-11-27 11:03:04 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:03:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643299  12 58449 39784485 A  ERRORB˜™š'
2018-11-27 11:03:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:03:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:03:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:03:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:03:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:03:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:03:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:03:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:03:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:03:55 [DEBUG   ] Finished updating board flags in 1.447 seconds
2018-11-27 11:04:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:04:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 11:04:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:04:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:04:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:04:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:04:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:04:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:04:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:04:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:04:55 [DEBUG   ] Finished updating board flags in 1.416 seconds
2018-11-27 11:05:13 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 11:05:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:05:49 [DEBUG   ] Got command RPT from MCS: ref #643308
2018-11-27 11:05:49 [DEBUG   ] summary = ERROR
2018-11-27 11:05:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643308  13 58449 39949013 A  ERRORERROR'
2018-11-27 11:05:49 [DEBUG   ] Got command RPT from MCS: ref #643309
2018-11-27 11:05:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:05:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643309  77 58449 39949014 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:05:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:05:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:05:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:05:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:05:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:05:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:05:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:05:55 [DEBUG   ] Finished updating board flags in 1.339 seconds
2018-11-27 11:06:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:06:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:06:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:06:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:06:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:06:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:06:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:06:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:06:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:06:55 [DEBUG   ] Finished updating board flags in 1.268 seconds
2018-11-27 11:07:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:07:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:07:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:07:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:07:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:07:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:07:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:07:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:07:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:07:56 [DEBUG   ] Finished updating board flags in 1.727 seconds
2018-11-27 11:08:00 [DEBUG   ] Got command DRX from MCS: ref #643333
2018-11-27 11:08:00 [INFO    ] sending drx command to lwa16
2018-11-27 11:08:00 [DEBUG   ] run_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 64500000.0 7 9 0']
2018-11-27 11:08:00 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 11:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   643333   8 58449 40080013 A  ERROR'
2018-11-27 11:08:00 [DEBUG   ] Got command DRX from MCS: ref #643334
2018-11-27 11:08:00 [INFO    ] sending drx command to lwa34
2018-11-27 11:08:00 [DEBUG   ] run_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 1 35100000.0 7 9 0']
2018-11-27 11:08:00 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 11:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   643334   8 58449 40080017 A  ERROR'
2018-11-27 11:08:00 [DEBUG   ] Got command DRX from MCS: ref #643335
2018-11-27 11:08:00 [INFO    ] sending drx command to lwa16
2018-11-27 11:08:00 [DEBUG   ] run_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 79200000.0 7 9 0']
2018-11-27 11:08:00 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 11:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   643335   8 58449 40080021 A  ERROR'
2018-11-27 11:08:00 [DEBUG   ] Got command DRX from MCS: ref #643336
2018-11-27 11:08:00 [INFO    ] sending drx command to lwa34
2018-11-27 11:08:00 [DEBUG   ] run_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 2 49800000.0 7 9 0']
2018-11-27 11:08:00 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 11:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   643336   8 58449 40080025 A  ERROR'
2018-11-27 11:08:00 [DEBUG   ] Got command BAM from MCS: ref #643338
2018-11-27 11:08:00 [INFO    ] Finished processing BAM request
2018-11-27 11:08:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643338   8 58449 40080029 A  ERROR'
2018-11-27 11:08:00 [DEBUG   ] Got command BAM from MCS: ref #643337
2018-11-27 11:08:00 [INFO    ] Finished processing BAM request
2018-11-27 11:08:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:08:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643337   8 58449 40080033 A  ERROR'
2018-11-27 11:08:00 [INFO    ] Finished starting DRX
2018-11-27 11:08:00 [INFO    ] Finished starting DRX
2018-11-27 11:08:00 [INFO    ] Finished starting DRX
2018-11-27 11:08:00 [INFO    ] Finished starting DRX
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:08:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:08:00 [DEBUG   ] spawn_process: at 2018-11-27 11:08:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:08:03 [DEBUG   ] Got command RPT from MCS: ref #643341
2018-11-27 11:08:03 [DEBUG   ] summary = ERROR
2018-11-27 11:08:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643341  13 58449 40083769 A  ERRORERROR'
2018-11-27 11:08:03 [DEBUG   ] Got command RPT from MCS: ref #643342
2018-11-27 11:08:03 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:08:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643342  77 58449 40083973 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:08:04 [DEBUG   ] Got command RPT from MCS: ref #643343
2018-11-27 11:08:04 [DEBUG   ] summary = ERROR
2018-11-27 11:08:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643343  13 58449 40084177 A  ERRORERROR'
2018-11-27 11:08:04 [DEBUG   ] Got command RPT from MCS: ref #643344
2018-11-27 11:08:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:08:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643344  77 58449 40084369 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:08:04 [DEBUG   ] Got command RPT from MCS: ref #643345
2018-11-27 11:08:04 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:08:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643345  12 58449 40084573 A  ERRORB˜™š'
2018-11-27 11:08:13 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 11:08:23 [DEBUG   ] Got command BAM from MCS: ref #643346
2018-11-27 11:08:23 [INFO    ] Finished processing BAM request
2018-11-27 11:08:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:08:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643346   8 58449 40103029 A  ERROR'
2018-11-27 11:08:23 [DEBUG   ] Got command BAM from MCS: ref #643347
2018-11-27 11:08:23 [INFO    ] Finished processing BAM request
2018-11-27 11:08:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:08:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643347   8 58449 40103033 A  ERROR'
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:08:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:08:23 [DEBUG   ] spawn_process: at 2018-11-27 11:08:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:08:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:08:46 [DEBUG   ] Got command BAM from MCS: ref #643349
2018-11-27 11:08:46 [INFO    ] Finished processing BAM request
2018-11-27 11:08:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:08:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643349   8 58449 40126033 A  ERROR'
2018-11-27 11:08:46 [DEBUG   ] Got command BAM from MCS: ref #643348
2018-11-27 11:08:46 [INFO    ] Finished processing BAM request
2018-11-27 11:08:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:08:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643348   8 58449 40126037 A  ERROR'
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:08:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:08:46 [DEBUG   ] spawn_process: at 2018-11-27 11:08:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:08:53 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:08:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:08:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:08:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:08:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:08:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:08:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:08:56 [DEBUG   ] Finished updating board flags in 1.621 seconds
2018-11-27 11:09:09 [DEBUG   ] Got command BAM from MCS: ref #643350
2018-11-27 11:09:09 [INFO    ] Finished processing BAM request
2018-11-27 11:09:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:09:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643350   8 58449 40149029 A  ERROR'
2018-11-27 11:09:09 [DEBUG   ] Got command BAM from MCS: ref #643351
2018-11-27 11:09:09 [INFO    ] Finished processing BAM request
2018-11-27 11:09:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:09:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643351   8 58449 40149033 A  ERROR'
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:09:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:09:09 [DEBUG   ] spawn_process: at 2018-11-27 11:09:09 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:09:13 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:09:32 [DEBUG   ] Got command BAM from MCS: ref #643353
2018-11-27 11:09:32 [INFO    ] Finished processing BAM request
2018-11-27 11:09:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:09:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643353   8 58449 40172029 A  ERROR'
2018-11-27 11:09:32 [DEBUG   ] Got command BAM from MCS: ref #643352
2018-11-27 11:09:32 [INFO    ] Finished processing BAM request
2018-11-27 11:09:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:09:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643352   8 58449 40172033 A  ERROR'
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:09:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:09:32 [DEBUG   ] spawn_process: at 2018-11-27 11:09:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:09:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:09:53 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:09:55 [DEBUG   ] Got command BAM from MCS: ref #643355
2018-11-27 11:09:55 [INFO    ] Finished processing BAM request
2018-11-27 11:09:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:09:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643355   8 58449 40195029 A  ERROR'
2018-11-27 11:09:55 [DEBUG   ] Got command BAM from MCS: ref #643354
2018-11-27 11:09:55 [INFO    ] Finished processing BAM request
2018-11-27 11:09:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:09:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643354   8 58449 40195033 A  ERROR'
2018-11-27 11:09:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:09:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:09:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:09:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:09:55 [WARNING ] (13, 1, 'bot')
2018-11-27 11:09:55 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:09:55 [DEBUG   ] Finished updating board flags in 0.988 seconds
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:09:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:09:55 [DEBUG   ] spawn_process: at 2018-11-27 11:09:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:10:13 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:10:18 [DEBUG   ] Got command BAM from MCS: ref #643361
2018-11-27 11:10:18 [INFO    ] Finished processing BAM request
2018-11-27 11:10:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:10:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643361   8 58449 40218029 A  ERROR'
2018-11-27 11:10:18 [DEBUG   ] Got command BAM from MCS: ref #643360
2018-11-27 11:10:18 [INFO    ] Finished processing BAM request
2018-11-27 11:10:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:10:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643360   8 58449 40218033 A  ERROR'
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:10:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:10:18 [DEBUG   ] spawn_process: at 2018-11-27 11:10:18 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:10:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:10:41 [DEBUG   ] Got command BAM from MCS: ref #643363
2018-11-27 11:10:41 [INFO    ] Finished processing BAM request
2018-11-27 11:10:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:10:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643363   8 58449 40241029 A  ERROR'
2018-11-27 11:10:41 [DEBUG   ] Got command BAM from MCS: ref #643362
2018-11-27 11:10:41 [INFO    ] Finished processing BAM request
2018-11-27 11:10:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:10:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643362   8 58449 40241033 A  ERROR'
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:10:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:10:41 [DEBUG   ] spawn_process: at 2018-11-27 11:10:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:10:49 [DEBUG   ] Got command RPT from MCS: ref #643364
2018-11-27 11:10:49 [DEBUG   ] summary = ERROR
2018-11-27 11:10:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643364  13 58449 40249013 A  ERRORERROR'
2018-11-27 11:10:49 [DEBUG   ] Got command RPT from MCS: ref #643365
2018-11-27 11:10:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:10:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643365  77 58449 40249014 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:10:53 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:10:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:10:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-27 11:10:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:10:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2018-11-27 11:10:56 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:10:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:10:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:10:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:10:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:10:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:10:56 [DEBUG   ] Finished updating board flags in 1.369 seconds
2018-11-27 11:11:04 [DEBUG   ] Got command BAM from MCS: ref #643381
2018-11-27 11:11:04 [INFO    ] Finished processing BAM request
2018-11-27 11:11:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:11:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643381   8 58449 40264033 A  ERROR'
2018-11-27 11:11:04 [DEBUG   ] Got command BAM from MCS: ref #643382
2018-11-27 11:11:04 [INFO    ] Finished processing BAM request
2018-11-27 11:11:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:11:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643382   8 58449 40264037 A  ERROR'
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:11:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:11:04 [DEBUG   ] spawn_process: at 2018-11-27 11:11:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:11:13 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:11:27 [DEBUG   ] Got command BAM from MCS: ref #643384
2018-11-27 11:11:27 [INFO    ] Finished processing BAM request
2018-11-27 11:11:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:11:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643384   8 58449 40287025 A  ERROR'
2018-11-27 11:11:27 [DEBUG   ] Got command BAM from MCS: ref #643383
2018-11-27 11:11:27 [INFO    ] Finished processing BAM request
2018-11-27 11:11:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:11:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643383   8 58449 40287029 A  ERROR'
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:11:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:11:27 [DEBUG   ] spawn_process: at 2018-11-27 11:11:27 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:11:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:11:50 [DEBUG   ] Got command BAM from MCS: ref #643385
2018-11-27 11:11:50 [INFO    ] Finished processing BAM request
2018-11-27 11:11:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:11:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643385   8 58449 40310029 A  ERROR'
2018-11-27 11:11:50 [DEBUG   ] Got command BAM from MCS: ref #643386
2018-11-27 11:11:50 [INFO    ] Finished processing BAM request
2018-11-27 11:11:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:11:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643386   8 58449 40310033 A  ERROR'
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:11:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:11:50 [DEBUG   ] spawn_process: at 2018-11-27 11:11:50 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:11:53 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:11:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:11:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 11:11:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 11:11:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:11:56 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:11:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 11:11:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:11:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:11:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:11:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:11:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:11:56 [DEBUG   ] Finished updating board flags in 1.192 seconds
2018-11-27 11:12:13 [DEBUG   ] Got command BAM from MCS: ref #643388
2018-11-27 11:12:13 [INFO    ] Finished processing BAM request
2018-11-27 11:12:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:12:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643388   8 58449 40333033 A  ERROR'
2018-11-27 11:12:13 [DEBUG   ] Got command BAM from MCS: ref #643387
2018-11-27 11:12:13 [INFO    ] Finished processing BAM request
2018-11-27 11:12:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:12:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643387   8 58449 40333041 A  ERROR'
2018-11-27 11:12:13 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:12:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:12:13 [DEBUG   ] spawn_process: at 2018-11-27 11:12:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:12:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:12:36 [DEBUG   ] Got command BAM from MCS: ref #643390
2018-11-27 11:12:36 [INFO    ] Finished processing BAM request
2018-11-27 11:12:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:12:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643390   8 58449 40356029 A  ERROR'
2018-11-27 11:12:36 [DEBUG   ] Got command BAM from MCS: ref #643389
2018-11-27 11:12:36 [INFO    ] Finished processing BAM request
2018-11-27 11:12:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:12:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643389   8 58449 40356033 A  ERROR'
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:12:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:12:36 [DEBUG   ] spawn_process: at 2018-11-27 11:12:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:12:53 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:12:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 11:12:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:12:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 11:12:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:12:56 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:12:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 11:12:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 11:12:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:12:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:12:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:12:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:12:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:12:56 [DEBUG   ] Finished updating board flags in 1.005 seconds
2018-11-27 11:12:59 [DEBUG   ] Got command BAM from MCS: ref #643394
2018-11-27 11:12:59 [INFO    ] Finished processing BAM request
2018-11-27 11:12:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:12:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643394   8 58449 40379029 A  ERROR'
2018-11-27 11:12:59 [DEBUG   ] Got command BAM from MCS: ref #643393
2018-11-27 11:12:59 [INFO    ] Finished processing BAM request
2018-11-27 11:12:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:12:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643393   8 58449 40379033 A  ERROR'
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:12:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:12:59 [DEBUG   ] spawn_process: at 2018-11-27 11:12:59 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:13:03 [DEBUG   ] Got command RPT from MCS: ref #643395
2018-11-27 11:13:03 [DEBUG   ] summary = ERROR
2018-11-27 11:13:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643395  13 58449 40383857 A  ERRORERROR'
2018-11-27 11:13:04 [DEBUG   ] Got command RPT from MCS: ref #643396
2018-11-27 11:13:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:13:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643396  77 58449 40384061 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:13:04 [DEBUG   ] Got command RPT from MCS: ref #643397
2018-11-27 11:13:04 [DEBUG   ] summary = ERROR
2018-11-27 11:13:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643397  13 58449 40384253 A  ERRORERROR'
2018-11-27 11:13:04 [DEBUG   ] Got command RPT from MCS: ref #643399
2018-11-27 11:13:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:13:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643399  77 58449 40384457 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:13:04 [DEBUG   ] Got command RPT from MCS: ref #643400
2018-11-27 11:13:04 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:13:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643400  12 58449 40384661 A  ERRORB¢ff'
2018-11-27 11:13:13 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:13:22 [DEBUG   ] Got command BAM from MCS: ref #643409
2018-11-27 11:13:22 [INFO    ] Finished processing BAM request
2018-11-27 11:13:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:13:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643409   8 58449 40402025 A  ERROR'
2018-11-27 11:13:22 [DEBUG   ] Got command BAM from MCS: ref #643410
2018-11-27 11:13:22 [INFO    ] Finished processing BAM request
2018-11-27 11:13:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:13:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643410   8 58449 40402029 A  ERROR'
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:13:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:13:22 [DEBUG   ] spawn_process: at 2018-11-27 11:13:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:13:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:13:45 [DEBUG   ] Got command BAM from MCS: ref #643412
2018-11-27 11:13:45 [INFO    ] Finished processing BAM request
2018-11-27 11:13:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:13:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643412   8 58449 40425029 A  ERROR'
2018-11-27 11:13:45 [DEBUG   ] Got command BAM from MCS: ref #643411
2018-11-27 11:13:45 [INFO    ] Finished processing BAM request
2018-11-27 11:13:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:13:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643411   8 58449 40425033 A  ERROR'
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:13:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:13:45 [DEBUG   ] spawn_process: at 2018-11-27 11:13:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:13:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 11:13:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:13:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:13:56 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:13:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:13:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:13:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:13:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:13:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:13:56 [DEBUG   ] Finished updating board flags in 1.029 seconds
2018-11-27 11:14:08 [DEBUG   ] Got command BAM from MCS: ref #643414
2018-11-27 11:14:08 [INFO    ] Finished processing BAM request
2018-11-27 11:14:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:14:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643414   8 58449 40448025 A  ERROR'
2018-11-27 11:14:08 [DEBUG   ] Got command BAM from MCS: ref #643413
2018-11-27 11:14:08 [INFO    ] Finished processing BAM request
2018-11-27 11:14:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:14:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643413   8 58449 40448029 A  ERROR'
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:14:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:14:08 [DEBUG   ] spawn_process: at 2018-11-27 11:14:08 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:14:13 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 11:14:31 [DEBUG   ] Got command BAM from MCS: ref #643416
2018-11-27 11:14:31 [INFO    ] Finished processing BAM request
2018-11-27 11:14:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:14:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643416   8 58449 40471025 A  ERROR'
2018-11-27 11:14:31 [DEBUG   ] Got command BAM from MCS: ref #643415
2018-11-27 11:14:31 [INFO    ] Finished processing BAM request
2018-11-27 11:14:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:14:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643415   8 58449 40471029 A  ERROR'
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:14:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:14:31 [DEBUG   ] spawn_process: at 2018-11-27 11:14:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:14:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:14:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 11:14:54 [DEBUG   ] Got command BAM from MCS: ref #643418
2018-11-27 11:14:54 [INFO    ] Finished processing BAM request
2018-11-27 11:14:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:14:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643418   8 58449 40494029 A  ERROR'
2018-11-27 11:14:54 [DEBUG   ] Got command BAM from MCS: ref #643417
2018-11-27 11:14:54 [INFO    ] Finished processing BAM request
2018-11-27 11:14:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:14:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643417   8 58449 40494033 A  ERROR'
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:14:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:14:54 [DEBUG   ] spawn_process: at 2018-11-27 11:14:54 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:14:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:14:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:14:56 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:14:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 11:14:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:14:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:14:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:14:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:14:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:14:56 [DEBUG   ] Finished updating board flags in 1.458 seconds
2018-11-27 11:15:13 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:15:17 [DEBUG   ] Got command BAM from MCS: ref #643419
2018-11-27 11:15:17 [INFO    ] Finished processing BAM request
2018-11-27 11:15:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:15:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643419   8 58449 40517025 A  ERROR'
2018-11-27 11:15:17 [DEBUG   ] Got command BAM from MCS: ref #643420
2018-11-27 11:15:17 [INFO    ] Finished processing BAM request
2018-11-27 11:15:17 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:15:17 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643420   8 58449 40517029 A  ERROR'
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:15:17 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:15:17 [DEBUG   ] spawn_process: at 2018-11-27 11:15:17 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:15:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:15:40 [DEBUG   ] Got command BAM from MCS: ref #643421
2018-11-27 11:15:40 [INFO    ] Finished processing BAM request
2018-11-27 11:15:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:15:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643421   8 58449 40540025 A  ERROR'
2018-11-27 11:15:40 [DEBUG   ] Got command BAM from MCS: ref #643422
2018-11-27 11:15:40 [INFO    ] Finished processing BAM request
2018-11-27 11:15:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:15:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643422   8 58449 40540029 A  ERROR'
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:15:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:15:40 [DEBUG   ] spawn_process: at 2018-11-27 11:15:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:15:49 [DEBUG   ] Got command RPT from MCS: ref #643423
2018-11-27 11:15:49 [DEBUG   ] summary = ERROR
2018-11-27 11:15:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643423  13 58449 40549017 A  ERRORERROR'
2018-11-27 11:15:49 [DEBUG   ] Got command RPT from MCS: ref #643424
2018-11-27 11:15:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:15:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643424  77 58449 40549018 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:15:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 11:15:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:15:56 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:15:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:15:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:15:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:15:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:15:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:15:56 [DEBUG   ] Finished updating board flags in 1.452 seconds
2018-11-27 11:16:03 [DEBUG   ] Got command BAM from MCS: ref #643440
2018-11-27 11:16:03 [INFO    ] Finished processing BAM request
2018-11-27 11:16:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:16:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643440   8 58449 40563033 A  ERROR'
2018-11-27 11:16:03 [DEBUG   ] Got command BAM from MCS: ref #643441
2018-11-27 11:16:03 [INFO    ] Finished processing BAM request
2018-11-27 11:16:03 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:16:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643441   8 58449 40563037 A  ERROR'
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:16:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:16:03 [DEBUG   ] spawn_process: at 2018-11-27 11:16:03 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:16:13 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:16:26 [DEBUG   ] Got command BAM from MCS: ref #643446
2018-11-27 11:16:26 [INFO    ] Finished processing BAM request
2018-11-27 11:16:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:16:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643446   8 58449 40586033 A  ERROR'
2018-11-27 11:16:26 [DEBUG   ] Got command BAM from MCS: ref #643447
2018-11-27 11:16:26 [INFO    ] Finished processing BAM request
2018-11-27 11:16:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:16:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643447   8 58449 40586037 A  ERROR'
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:16:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:16:26 [DEBUG   ] spawn_process: at 2018-11-27 11:16:26 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:16:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:16:49 [DEBUG   ] Got command BAM from MCS: ref #643449
2018-11-27 11:16:49 [INFO    ] Finished processing BAM request
2018-11-27 11:16:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:16:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643449   8 58449 40609025 A  ERROR'
2018-11-27 11:16:49 [DEBUG   ] Got command BAM from MCS: ref #643448
2018-11-27 11:16:49 [INFO    ] Finished processing BAM request
2018-11-27 11:16:49 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:16:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643448   8 58449 40609029 A  ERROR'
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:16:49 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:16:49 [DEBUG   ] spawn_process: at 2018-11-27 11:16:49 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:16:53 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:16:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:16:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:16:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:16:56 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:16:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 11:16:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:16:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:16:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:16:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:16:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:16:56 [DEBUG   ] Finished updating board flags in 1.408 seconds
2018-11-27 11:17:12 [DEBUG   ] Got command BAM from MCS: ref #643450
2018-11-27 11:17:12 [INFO    ] Finished processing BAM request
2018-11-27 11:17:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:17:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643450   8 58449 40632025 A  ERROR'
2018-11-27 11:17:12 [DEBUG   ] Got command BAM from MCS: ref #643451
2018-11-27 11:17:12 [INFO    ] Finished processing BAM request
2018-11-27 11:17:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:17:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643451   8 58449 40632029 A  ERROR'
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:17:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:17:12 [DEBUG   ] spawn_process: at 2018-11-27 11:17:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:17:13 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:17:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:17:35 [DEBUG   ] Got command BAM from MCS: ref #643452
2018-11-27 11:17:35 [INFO    ] Finished processing BAM request
2018-11-27 11:17:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:17:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643452   8 58449 40655025 A  ERROR'
2018-11-27 11:17:35 [DEBUG   ] Got command BAM from MCS: ref #643453
2018-11-27 11:17:35 [INFO    ] Finished processing BAM request
2018-11-27 11:17:35 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:17:35 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643453   8 58449 40655029 A  ERROR'
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:17:35 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:17:35 [DEBUG   ] spawn_process: at 2018-11-27 11:17:35 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:17:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 11:17:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:17:56 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:17:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:17:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:17:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:17:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:17:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:17:56 [DEBUG   ] Finished updating board flags in 1.304 seconds
2018-11-27 11:17:58 [DEBUG   ] Got command BAM from MCS: ref #643454
2018-11-27 11:17:58 [INFO    ] Finished processing BAM request
2018-11-27 11:17:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643454   8 58449 40678033 A  ERROR'
2018-11-27 11:17:58 [DEBUG   ] Got command BAM from MCS: ref #643455
2018-11-27 11:17:58 [INFO    ] Finished processing BAM request
2018-11-27 11:17:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:17:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643455   8 58449 40678037 A  ERROR'
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:17:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:17:58 [DEBUG   ] spawn_process: at 2018-11-27 11:17:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:18:03 [DEBUG   ] Got command RPT from MCS: ref #643456
2018-11-27 11:18:03 [DEBUG   ] summary = ERROR
2018-11-27 11:18:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643456  13 58449 40683945 A  ERRORERROR'
2018-11-27 11:18:04 [DEBUG   ] Got command RPT from MCS: ref #643457
2018-11-27 11:18:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:18:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643457  77 58449 40684137 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:18:04 [DEBUG   ] Got command RPT from MCS: ref #643458
2018-11-27 11:18:04 [DEBUG   ] summary = ERROR
2018-11-27 11:18:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643458  13 58449 40684341 A  ERRORERROR'
2018-11-27 11:18:04 [DEBUG   ] Got command RPT from MCS: ref #643459
2018-11-27 11:18:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:18:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643459  77 58449 40684545 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:18:04 [DEBUG   ] Got command RPT from MCS: ref #643460
2018-11-27 11:18:04 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:18:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643460  12 58449 40684749 A  ERRORB¢ff'
2018-11-27 11:18:13 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:18:21 [DEBUG   ] Got command BAM from MCS: ref #643462
2018-11-27 11:18:21 [INFO    ] Finished processing BAM request
2018-11-27 11:18:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:18:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643462   8 58449 40701033 A  ERROR'
2018-11-27 11:18:21 [DEBUG   ] Got command BAM from MCS: ref #643461
2018-11-27 11:18:21 [INFO    ] Finished processing BAM request
2018-11-27 11:18:21 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:18:21 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643461   8 58449 40701037 A  ERROR'
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:18:21 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:18:21 [DEBUG   ] spawn_process: at 2018-11-27 11:18:21 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:18:33 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:18:44 [DEBUG   ] Got command BAM from MCS: ref #643463
2018-11-27 11:18:44 [INFO    ] Finished processing BAM request
2018-11-27 11:18:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:18:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643463   8 58449 40724025 A  ERROR'
2018-11-27 11:18:44 [DEBUG   ] Got command BAM from MCS: ref #643464
2018-11-27 11:18:44 [INFO    ] Finished processing BAM request
2018-11-27 11:18:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:18:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643464   8 58449 40724029 A  ERROR'
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:18:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:18:44 [DEBUG   ] spawn_process: at 2018-11-27 11:18:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:18:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 11:18:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 11:18:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:18:57 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:18:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 11:18:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:18:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:18:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:18:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:18:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:18:57 [DEBUG   ] Finished updating board flags in 1.776 seconds
2018-11-27 11:19:07 [DEBUG   ] Got command BAM from MCS: ref #643466
2018-11-27 11:19:07 [INFO    ] Finished processing BAM request
2018-11-27 11:19:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:19:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643466   8 58449 40747025 A  ERROR'
2018-11-27 11:19:07 [DEBUG   ] Got command BAM from MCS: ref #643465
2018-11-27 11:19:07 [INFO    ] Finished processing BAM request
2018-11-27 11:19:07 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:19:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643465   8 58449 40747029 A  ERROR'
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:19:07 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:19:07 [DEBUG   ] spawn_process: at 2018-11-27 11:19:07 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:19:13 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 11:19:30 [DEBUG   ] Got command BAM from MCS: ref #643472
2018-11-27 11:19:30 [INFO    ] Finished processing BAM request
2018-11-27 11:19:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:19:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643472   8 58449 40770033 A  ERROR'
2018-11-27 11:19:30 [DEBUG   ] Got command BAM from MCS: ref #643471
2018-11-27 11:19:30 [INFO    ] Finished processing BAM request
2018-11-27 11:19:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:19:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643471   8 58449 40770037 A  ERROR'
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:19:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:19:30 [DEBUG   ] spawn_process: at 2018-11-27 11:19:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:19:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:19:53 [DEBUG   ] Got command BAM from MCS: ref #643474
2018-11-27 11:19:53 [INFO    ] Finished processing BAM request
2018-11-27 11:19:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:19:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643474   8 58449 40793029 A  ERROR'
2018-11-27 11:19:53 [DEBUG   ] Got command BAM from MCS: ref #643473
2018-11-27 11:19:53 [INFO    ] Finished processing BAM request
2018-11-27 11:19:53 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:19:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643473   8 58449 40793033 A  ERROR'
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:19:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:19:53 [DEBUG   ] spawn_process: at 2018-11-27 11:19:53 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:19:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:19:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:19:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 11:19:56 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:19:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:19:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:19:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:19:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:19:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:19:56 [DEBUG   ] Finished updating board flags in 1.020 seconds
2018-11-27 11:20:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:20:16 [DEBUG   ] Got command BAM from MCS: ref #643476
2018-11-27 11:20:16 [INFO    ] Finished processing BAM request
2018-11-27 11:20:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:20:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643476   8 58449 40816029 A  ERROR'
2018-11-27 11:20:16 [DEBUG   ] Got command BAM from MCS: ref #643475
2018-11-27 11:20:16 [INFO    ] Finished processing BAM request
2018-11-27 11:20:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:20:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643475   8 58449 40816033 A  ERROR'
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:20:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:20:16 [DEBUG   ] spawn_process: at 2018-11-27 11:20:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:20:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:20:39 [DEBUG   ] Got command BAM from MCS: ref #643477
2018-11-27 11:20:39 [INFO    ] Finished processing BAM request
2018-11-27 11:20:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:20:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643477   8 58449 40839025 A  ERROR'
2018-11-27 11:20:39 [DEBUG   ] Got command BAM from MCS: ref #643478
2018-11-27 11:20:39 [INFO    ] Finished processing BAM request
2018-11-27 11:20:39 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:20:39 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643478   8 58449 40839029 A  ERROR'
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:20:39 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:20:39 [DEBUG   ] spawn_process: at 2018-11-27 11:20:39 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:20:49 [DEBUG   ] Got command RPT from MCS: ref #643479
2018-11-27 11:20:49 [DEBUG   ] summary = ERROR
2018-11-27 11:20:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643479  13 58449 40849009 A  ERRORERROR'
2018-11-27 11:20:49 [DEBUG   ] Got command RPT from MCS: ref #643480
2018-11-27 11:20:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:20:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643480  77 58449 40849010 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:20:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 11:20:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:20:56 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:20:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 11:20:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 11:20:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:20:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:20:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:20:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:20:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:20:56 [DEBUG   ] Finished updating board flags in 1.025 seconds
2018-11-27 11:21:02 [DEBUG   ] Got command BAM from MCS: ref #643497
2018-11-27 11:21:02 [INFO    ] Finished processing BAM request
2018-11-27 11:21:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:21:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643497   8 58449 40862029 A  ERROR'
2018-11-27 11:21:02 [DEBUG   ] Got command BAM from MCS: ref #643496
2018-11-27 11:21:02 [INFO    ] Finished processing BAM request
2018-11-27 11:21:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:21:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643496   8 58449 40862033 A  ERROR'
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:21:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:21:02 [DEBUG   ] spawn_process: at 2018-11-27 11:21:02 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:21:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:21:25 [DEBUG   ] Got command BAM from MCS: ref #643499
2018-11-27 11:21:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:21:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643499   8 58449 40885078 A  ERROR'
2018-11-27 11:21:25 [DEBUG   ] Got command BAM from MCS: ref #643498
2018-11-27 11:21:25 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:21:25 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643498   8 58449 40885082 A  ERROR'
2018-11-27 11:21:25 [INFO    ] Finished processing BAM request
2018-11-27 11:21:25 [INFO    ] Finished processing BAM request
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:21:25 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:21:25 [DEBUG   ] spawn_process: at 2018-11-27 11:21:25 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:21:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:21:48 [DEBUG   ] Got command BAM from MCS: ref #643500
2018-11-27 11:21:48 [INFO    ] Finished processing BAM request
2018-11-27 11:21:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:21:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643500   8 58449 40908030 A  ERROR'
2018-11-27 11:21:48 [DEBUG   ] Got command BAM from MCS: ref #643501
2018-11-27 11:21:48 [INFO    ] Finished processing BAM request
2018-11-27 11:21:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:21:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643501   8 58449 40908034 A  ERROR'
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:21:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:21:48 [DEBUG   ] spawn_process: at 2018-11-27 11:21:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:21:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:21:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:21:56 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 11:21:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:21:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:21:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:21:56 [WARNING ] (13, 1, 'bot')
2018-11-27 11:21:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:21:56 [DEBUG   ] Finished updating board flags in 0.994 seconds
2018-11-27 11:22:11 [DEBUG   ] Got command BAM from MCS: ref #643503
2018-11-27 11:22:11 [INFO    ] Finished processing BAM request
2018-11-27 11:22:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:22:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643503   8 58449 40931030 A  ERROR'
2018-11-27 11:22:11 [DEBUG   ] Got command BAM from MCS: ref #643502
2018-11-27 11:22:11 [INFO    ] Finished processing BAM request
2018-11-27 11:22:11 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:22:11 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643502   8 58449 40931034 A  ERROR'
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:22:11 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:22:11 [DEBUG   ] spawn_process: at 2018-11-27 11:22:11 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:22:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:22:34 [DEBUG   ] Got command BAM from MCS: ref #643509
2018-11-27 11:22:34 [INFO    ] Finished processing BAM request
2018-11-27 11:22:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:22:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643509   8 58449 40954029 A  ERROR'
2018-11-27 11:22:34 [DEBUG   ] Got command BAM from MCS: ref #643508
2018-11-27 11:22:34 [INFO    ] Finished processing BAM request
2018-11-27 11:22:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:22:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643508   8 58449 40954033 A  ERROR'
2018-11-27 11:22:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:22:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:22:34 [DEBUG   ] spawn_process: at 2018-11-27 11:22:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:22:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:22:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 11:22:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 11:22:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:22:57 [DEBUG   ] Got command BAM from MCS: ref #643512
2018-11-27 11:22:57 [INFO    ] Finished processing BAM request
2018-11-27 11:22:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:22:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643512   8 58449 40977029 A  ERROR'
2018-11-27 11:22:57 [DEBUG   ] Got command BAM from MCS: ref #643513
2018-11-27 11:22:57 [INFO    ] Finished processing BAM request
2018-11-27 11:22:57 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:22:57 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643513   8 58449 40977033 A  ERROR'
2018-11-27 11:22:57 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 11:22:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:22:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:22:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:22:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:22:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:22:57 [DEBUG   ] Finished updating board flags in 1.356 seconds
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:22:57 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:22:57 [DEBUG   ] spawn_process: at 2018-11-27 11:22:57 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:23:04 [DEBUG   ] Got command RPT from MCS: ref #643514
2018-11-27 11:23:04 [DEBUG   ] summary = ERROR
2018-11-27 11:23:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643514  13 58449 40984029 A  ERRORERROR'
2018-11-27 11:23:04 [DEBUG   ] Got command RPT from MCS: ref #643515
2018-11-27 11:23:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:23:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643515  77 58449 40984221 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:23:04 [DEBUG   ] Got command RPT from MCS: ref #643516
2018-11-27 11:23:04 [DEBUG   ] summary = ERROR
2018-11-27 11:23:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643516  13 58449 40984425 A  ERRORERROR'
2018-11-27 11:23:04 [DEBUG   ] Got command RPT from MCS: ref #643518
2018-11-27 11:23:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:23:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643518  77 58449 40984629 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:23:04 [DEBUG   ] Got command RPT from MCS: ref #643519
2018-11-27 11:23:04 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:23:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643519  12 58449 40984833 A  ERRORB£ff'
2018-11-27 11:23:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:23:20 [DEBUG   ] Got command BAM from MCS: ref #643525
2018-11-27 11:23:20 [INFO    ] Finished processing BAM request
2018-11-27 11:23:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:23:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643525   8 58449 41000025 A  ERROR'
2018-11-27 11:23:20 [DEBUG   ] Got command BAM from MCS: ref #643524
2018-11-27 11:23:20 [INFO    ] Finished processing BAM request
2018-11-27 11:23:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:23:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643524   8 58449 41000029 A  ERROR'
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:23:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:23:20 [DEBUG   ] spawn_process: at 2018-11-27 11:23:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:23:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 11:23:43 [DEBUG   ] Got command BAM from MCS: ref #643526
2018-11-27 11:23:43 [INFO    ] Finished processing BAM request
2018-11-27 11:23:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:23:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643526   8 58449 41023029 A  ERROR'
2018-11-27 11:23:43 [DEBUG   ] Got command BAM from MCS: ref #643527
2018-11-27 11:23:43 [INFO    ] Finished processing BAM request
2018-11-27 11:23:43 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:23:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643527   8 58449 41023033 A  ERROR'
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:23:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:23:43 [DEBUG   ] spawn_process: at 2018-11-27 11:23:43 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:23:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:23:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:23:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 11:23:57 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 11:23:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:23:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:23:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:23:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:23:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:23:57 [DEBUG   ] Finished updating board flags in 1.192 seconds
2018-11-27 11:24:06 [DEBUG   ] Got command BAM from MCS: ref #643528
2018-11-27 11:24:06 [INFO    ] Finished processing BAM request
2018-11-27 11:24:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:24:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643528   8 58449 41046025 A  ERROR'
2018-11-27 11:24:06 [DEBUG   ] Got command BAM from MCS: ref #643529
2018-11-27 11:24:06 [INFO    ] Finished processing BAM request
2018-11-27 11:24:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:24:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643529   8 58449 41046029 A  ERROR'
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:24:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:24:06 [DEBUG   ] spawn_process: at 2018-11-27 11:24:06 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:24:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:24:29 [DEBUG   ] Got command BAM from MCS: ref #643531
2018-11-27 11:24:29 [INFO    ] Finished processing BAM request
2018-11-27 11:24:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:24:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643531   8 58449 41069029 A  ERROR'
2018-11-27 11:24:29 [DEBUG   ] Got command BAM from MCS: ref #643530
2018-11-27 11:24:29 [INFO    ] Finished processing BAM request
2018-11-27 11:24:29 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:24:29 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643530   8 58449 41069033 A  ERROR'
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:24:29 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:24:29 [DEBUG   ] spawn_process: at 2018-11-27 11:24:29 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:24:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:24:52 [DEBUG   ] Got command BAM from MCS: ref #643533
2018-11-27 11:24:52 [INFO    ] Finished processing BAM request
2018-11-27 11:24:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:24:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643533   8 58449 41092033 A  ERROR'
2018-11-27 11:24:52 [DEBUG   ] Got command BAM from MCS: ref #643532
2018-11-27 11:24:52 [INFO    ] Finished processing BAM request
2018-11-27 11:24:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:24:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643532   8 58449 41092037 A  ERROR'
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:24:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:24:52 [DEBUG   ] spawn_process: at 2018-11-27 11:24:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:24:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:24:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 11:24:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:24:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2018-11-27 11:24:57 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:24:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 11:24:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:24:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:24:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:24:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:24:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:24:57 [DEBUG   ] Finished updating board flags in 0.999 seconds
2018-11-27 11:25:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:25:15 [DEBUG   ] Got command BAM from MCS: ref #643534
2018-11-27 11:25:15 [INFO    ] Finished processing BAM request
2018-11-27 11:25:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:25:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643534   8 58449 41115029 A  ERROR'
2018-11-27 11:25:15 [DEBUG   ] Got command BAM from MCS: ref #643535
2018-11-27 11:25:15 [INFO    ] Finished processing BAM request
2018-11-27 11:25:15 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:25:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643535   8 58449 41115033 A  ERROR'
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:25:15 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:25:15 [DEBUG   ] spawn_process: at 2018-11-27 11:25:15 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:25:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 11:25:38 [DEBUG   ] Got command BAM from MCS: ref #643541
2018-11-27 11:25:38 [INFO    ] Finished processing BAM request
2018-11-27 11:25:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:25:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643541   8 58449 41138029 A  ERROR'
2018-11-27 11:25:38 [DEBUG   ] Got command BAM from MCS: ref #643540
2018-11-27 11:25:38 [INFO    ] Finished processing BAM request
2018-11-27 11:25:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:25:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643540   8 58449 41138033 A  ERROR'
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:25:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:25:38 [DEBUG   ] spawn_process: at 2018-11-27 11:25:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:25:49 [DEBUG   ] Got command RPT from MCS: ref #643542
2018-11-27 11:25:49 [DEBUG   ] summary = ERROR
2018-11-27 11:25:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643542  13 58449 41149013 A  ERRORERROR'
2018-11-27 11:25:49 [DEBUG   ] Got command RPT from MCS: ref #643543
2018-11-27 11:25:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:25:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643543  77 58449 41149014 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:25:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:25:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:25:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 11:25:57 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:25:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:25:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:25:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:25:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:25:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:25:57 [DEBUG   ] Finished updating board flags in 1.283 seconds
2018-11-27 11:26:01 [DEBUG   ] Got command BAM from MCS: ref #643559
2018-11-27 11:26:01 [INFO    ] Finished processing BAM request
2018-11-27 11:26:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:26:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643559   8 58449 41161033 A  ERROR'
2018-11-27 11:26:01 [DEBUG   ] Got command BAM from MCS: ref #643560
2018-11-27 11:26:01 [INFO    ] Finished processing BAM request
2018-11-27 11:26:01 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:26:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643560   8 58449 41161037 A  ERROR'
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:26:01 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:26:01 [DEBUG   ] spawn_process: at 2018-11-27 11:26:01 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:26:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:26:24 [DEBUG   ] Got command BAM from MCS: ref #643562
2018-11-27 11:26:24 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:26:24 [INFO    ] Finished processing BAM request
2018-11-27 11:26:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643562   8 58449 41184029 A  ERROR'
2018-11-27 11:26:24 [DEBUG   ] Got command BAM from MCS: ref #643561
2018-11-27 11:26:24 [INFO    ] Finished processing BAM request
2018-11-27 11:26:24 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:26:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643561   8 58449 41184033 A  ERROR'
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:26:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:26:24 [DEBUG   ] spawn_process: at 2018-11-27 11:26:24 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:26:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:26:47 [DEBUG   ] Got command BAM from MCS: ref #643563
2018-11-27 11:26:47 [INFO    ] Finished processing BAM request
2018-11-27 11:26:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:26:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643563   8 58449 41207029 A  ERROR'
2018-11-27 11:26:47 [DEBUG   ] Got command BAM from MCS: ref #643564
2018-11-27 11:26:47 [INFO    ] Finished processing BAM request
2018-11-27 11:26:47 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:26:47 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643564   8 58449 41207033 A  ERROR'
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:26:47 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:26:47 [DEBUG   ] spawn_process: at 2018-11-27 11:26:47 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:26:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:26:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:26:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2018-11-27 11:26:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:26:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:26:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:26:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:26:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:26:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:26:58 [DEBUG   ] Finished updating board flags in 1.780 seconds
2018-11-27 11:27:10 [DEBUG   ] Got command BAM from MCS: ref #643565
2018-11-27 11:27:10 [INFO    ] Finished processing BAM request
2018-11-27 11:27:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:27:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643565   8 58449 41230029 A  ERROR'
2018-11-27 11:27:10 [DEBUG   ] Got command BAM from MCS: ref #643566
2018-11-27 11:27:10 [INFO    ] Finished processing BAM request
2018-11-27 11:27:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:27:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643566   8 58449 41230033 A  ERROR'
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:27:10 [DEBUG   ] spawn_process: at 2018-11-27 11:27:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:27:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:27:33 [DEBUG   ] Got command BAM from MCS: ref #643568
2018-11-27 11:27:33 [INFO    ] Finished processing BAM request
2018-11-27 11:27:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:27:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643568   8 58449 41253045 A  ERROR'
2018-11-27 11:27:33 [DEBUG   ] Got command BAM from MCS: ref #643567
2018-11-27 11:27:33 [INFO    ] Finished processing BAM request
2018-11-27 11:27:33 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:27:33 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643567   8 58449 41253050 A  ERROR'
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:27:33 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:27:33 [DEBUG   ] spawn_process: at 2018-11-27 11:27:33 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:27:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 11:27:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:27:56 [DEBUG   ] Got command BAM from MCS: ref #643570
2018-11-27 11:27:56 [INFO    ] Finished processing BAM request
2018-11-27 11:27:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:27:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643570   8 58449 41276029 A  ERROR'
2018-11-27 11:27:56 [DEBUG   ] Got command BAM from MCS: ref #643569
2018-11-27 11:27:56 [INFO    ] Finished processing BAM request
2018-11-27 11:27:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:27:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643569   8 58449 41276033 A  ERROR'
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:27:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:27:56 [DEBUG   ] spawn_process: at 2018-11-27 11:27:56 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:27:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2018-11-27 11:27:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 11:27:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:27:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:27:57 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:27:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:27:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:27:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:27:57 [WARNING ] (13, 1, 'bot')
2018-11-27 11:27:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:27:57 [DEBUG   ] Finished updating board flags in 0.984 seconds
2018-11-27 11:28:04 [DEBUG   ] Got command RPT from MCS: ref #643571
2018-11-27 11:28:04 [DEBUG   ] summary = ERROR
2018-11-27 11:28:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643571  13 58449 41284117 A  ERRORERROR'
2018-11-27 11:28:04 [DEBUG   ] Got command RPT from MCS: ref #643572
2018-11-27 11:28:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:28:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643572  77 58449 41284315 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:28:04 [DEBUG   ] Got command RPT from MCS: ref #643573
2018-11-27 11:28:04 [DEBUG   ] summary = ERROR
2018-11-27 11:28:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643573  13 58449 41284517 A  ERRORERROR'
2018-11-27 11:28:04 [DEBUG   ] Got command RPT from MCS: ref #643574
2018-11-27 11:28:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:28:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643574  77 58449 41284709 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:28:04 [DEBUG   ] Got command RPT from MCS: ref #643575
2018-11-27 11:28:04 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:28:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643575  12 58449 41284913 A  ERRORB¢ff'
2018-11-27 11:28:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 11:28:19 [DEBUG   ] Got command BAM from MCS: ref #643581
2018-11-27 11:28:19 [INFO    ] Finished processing BAM request
2018-11-27 11:28:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:28:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643581   8 58449 41299033 A  ERROR'
2018-11-27 11:28:19 [DEBUG   ] Got command BAM from MCS: ref #643580
2018-11-27 11:28:19 [INFO    ] Finished processing BAM request
2018-11-27 11:28:19 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:28:19 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643580   8 58449 41299037 A  ERROR'
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:28:19 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:28:19 [DEBUG   ] spawn_process: at 2018-11-27 11:28:19 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:28:34 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 11:28:42 [DEBUG   ] Got command BAM from MCS: ref #643583
2018-11-27 11:28:42 [INFO    ] Finished processing BAM request
2018-11-27 11:28:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:28:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643583   8 58449 41322033 A  ERROR'
2018-11-27 11:28:42 [DEBUG   ] Got command BAM from MCS: ref #643582
2018-11-27 11:28:42 [INFO    ] Finished processing BAM request
2018-11-27 11:28:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:28:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643582   8 58449 41322037 A  ERROR'
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:28:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:28:42 [DEBUG   ] spawn_process: at 2018-11-27 11:28:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:28:54 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:28:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:28:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:28:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:28:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:28:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:28:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:28:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:28:58 [DEBUG   ] Finished updating board flags in 1.397 seconds
2018-11-27 11:29:05 [DEBUG   ] Got command BAM from MCS: ref #643585
2018-11-27 11:29:05 [INFO    ] Finished processing BAM request
2018-11-27 11:29:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:29:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643585   8 58449 41345033 A  ERROR'
2018-11-27 11:29:05 [DEBUG   ] Got command BAM from MCS: ref #643584
2018-11-27 11:29:05 [INFO    ] Finished processing BAM request
2018-11-27 11:29:05 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:29:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643584   8 58449 41345037 A  ERROR'
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:29:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:29:05 [DEBUG   ] spawn_process: at 2018-11-27 11:29:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:29:14 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:29:28 [DEBUG   ] Got command BAM from MCS: ref #643587
2018-11-27 11:29:28 [INFO    ] Finished processing BAM request
2018-11-27 11:29:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:29:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643587   8 58449 41368025 A  ERROR'
2018-11-27 11:29:28 [DEBUG   ] Got command BAM from MCS: ref #643586
2018-11-27 11:29:28 [INFO    ] Finished processing BAM request
2018-11-27 11:29:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:29:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643586   8 58449 41368029 A  ERROR'
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:29:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:29:28 [DEBUG   ] spawn_process: at 2018-11-27 11:29:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:29:34 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:29:51 [DEBUG   ] Got command BAM from MCS: ref #643589
2018-11-27 11:29:51 [INFO    ] Finished processing BAM request
2018-11-27 11:29:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:29:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643589   8 58449 41391025 A  ERROR'
2018-11-27 11:29:51 [DEBUG   ] Got command BAM from MCS: ref #643588
2018-11-27 11:29:51 [INFO    ] Finished processing BAM request
2018-11-27 11:29:51 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:29:51 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643588   8 58449 41391029 A  ERROR'
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:29:51 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:29:51 [DEBUG   ] spawn_process: at 2018-11-27 11:29:51 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:29:54 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:29:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:29:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 11:29:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:29:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:29:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:29:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:29:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:29:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:29:58 [DEBUG   ] Finished updating board flags in 1.204 seconds
2018-11-27 11:30:14 [DEBUG   ] Got command BAM from MCS: ref #643591
2018-11-27 11:30:14 [INFO    ] Finished processing BAM request
2018-11-27 11:30:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:30:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643591   8 58449 41414025 A  ERROR'
2018-11-27 11:30:14 [DEBUG   ] Got command BAM from MCS: ref #643590
2018-11-27 11:30:14 [INFO    ] Finished processing BAM request
2018-11-27 11:30:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:30:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643590   8 58449 41414029 A  ERROR'
2018-11-27 11:30:14 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:30:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:30:14 [DEBUG   ] spawn_process: at 2018-11-27 11:30:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:30:34 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:30:37 [DEBUG   ] Got command BAM from MCS: ref #643593
2018-11-27 11:30:37 [INFO    ] Finished processing BAM request
2018-11-27 11:30:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:30:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643593   8 58449 41437025 A  ERROR'
2018-11-27 11:30:37 [DEBUG   ] Got command BAM from MCS: ref #643592
2018-11-27 11:30:37 [INFO    ] Finished processing BAM request
2018-11-27 11:30:37 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:30:37 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643592   8 58449 41437029 A  ERROR'
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:30:37 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:30:37 [DEBUG   ] spawn_process: at 2018-11-27 11:30:37 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:30:49 [DEBUG   ] Got command RPT from MCS: ref #643594
2018-11-27 11:30:49 [DEBUG   ] summary = ERROR
2018-11-27 11:30:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643594  13 58449 41449017 A  ERRORERROR'
2018-11-27 11:30:49 [DEBUG   ] Got command RPT from MCS: ref #643595
2018-11-27 11:30:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:30:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643595  77 58449 41449018 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:30:54 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:30:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:30:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:30:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 11:30:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:30:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:30:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:30:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:30:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:30:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:30:58 [DEBUG   ] Finished updating board flags in 1.177 seconds
2018-11-27 11:31:00 [DEBUG   ] Got command BAM from MCS: ref #643611
2018-11-27 11:31:00 [INFO    ] Finished processing BAM request
2018-11-27 11:31:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:31:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643611   8 58449 41460033 A  ERROR'
2018-11-27 11:31:00 [DEBUG   ] Got command BAM from MCS: ref #643612
2018-11-27 11:31:00 [INFO    ] Finished processing BAM request
2018-11-27 11:31:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:31:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643612   8 58449 41460037 A  ERROR'
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:31:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:31:00 [DEBUG   ] spawn_process: at 2018-11-27 11:31:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:31:14 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:31:23 [DEBUG   ] Got command BAM from MCS: ref #643618
2018-11-27 11:31:23 [INFO    ] Finished processing BAM request
2018-11-27 11:31:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:31:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643618   8 58449 41483033 A  ERROR'
2018-11-27 11:31:23 [DEBUG   ] Got command BAM from MCS: ref #643617
2018-11-27 11:31:23 [INFO    ] Finished processing BAM request
2018-11-27 11:31:23 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:31:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643617   8 58449 41483037 A  ERROR'
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:31:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:31:23 [DEBUG   ] spawn_process: at 2018-11-27 11:31:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:31:34 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:31:46 [DEBUG   ] Got command BAM from MCS: ref #643619
2018-11-27 11:31:46 [INFO    ] Finished processing BAM request
2018-11-27 11:31:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:31:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643619   8 58449 41506025 A  ERROR'
2018-11-27 11:31:46 [DEBUG   ] Got command BAM from MCS: ref #643620
2018-11-27 11:31:46 [INFO    ] Finished processing BAM request
2018-11-27 11:31:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:31:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643620   8 58449 41506029 A  ERROR'
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 11:31:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 11:31:46 [DEBUG   ] spawn_process: at 2018-11-27 11:31:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 11:31:54 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:31:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:31:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 11:31:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 5
2018-11-27 11:31:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:31:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 11:31:58 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 11:31:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:31:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:31:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:31:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:31:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:31:58 [DEBUG   ] Finished updating board flags in 1.012 seconds
2018-11-27 11:32:09 [DEBUG   ] Got command BAM from MCS: ref #643622
2018-11-27 11:32:09 [INFO    ] Finished processing BAM request
2018-11-27 11:32:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:32:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643622   8 58449 41529025 A  ERROR'
2018-11-27 11:32:09 [DEBUG   ] Got command BAM from MCS: ref #643621
2018-11-27 11:32:09 [INFO    ] Finished processing BAM request
2018-11-27 11:32:09 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:32:09 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643621   8 58449 41529029 A  ERROR'
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:32:09 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:32:09 [DEBUG   ] spawn_process: at 2018-11-27 11:32:09 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:32:14 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 11:32:32 [DEBUG   ] Got command BAM from MCS: ref #643624
2018-11-27 11:32:32 [INFO    ] Finished processing BAM request
2018-11-27 11:32:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:32:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643624   8 58449 41552033 A  ERROR'
2018-11-27 11:32:32 [DEBUG   ] Got command BAM from MCS: ref #643623
2018-11-27 11:32:32 [INFO    ] Finished processing BAM request
2018-11-27 11:32:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:32:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643623   8 58449 41552037 A  ERROR'
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:32:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:32:32 [DEBUG   ] spawn_process: at 2018-11-27 11:32:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:32:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:32:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 11:32:55 [DEBUG   ] Got command BAM from MCS: ref #643628
2018-11-27 11:32:55 [INFO    ] Finished processing BAM request
2018-11-27 11:32:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:32:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643628   8 58449 41575029 A  ERROR'
2018-11-27 11:32:55 [DEBUG   ] Got command BAM from MCS: ref #643627
2018-11-27 11:32:55 [INFO    ] Finished processing BAM request
2018-11-27 11:32:55 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:32:55 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643627   8 58449 41575033 A  ERROR'
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:32:55 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:32:55 [DEBUG   ] spawn_process: at 2018-11-27 11:32:55 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:32:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 11:32:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:32:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:32:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:32:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:32:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:32:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:32:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:32:58 [DEBUG   ] Finished updating board flags in 0.982 seconds
2018-11-27 11:33:04 [DEBUG   ] Got command RPT from MCS: ref #643629
2018-11-27 11:33:04 [DEBUG   ] summary = ERROR
2018-11-27 11:33:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643629  13 58449 41584201 A  ERRORERROR'
2018-11-27 11:33:04 [DEBUG   ] Got command RPT from MCS: ref #643630
2018-11-27 11:33:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:33:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643630  77 58449 41584416 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:33:04 [DEBUG   ] Got command RPT from MCS: ref #643631
2018-11-27 11:33:04 [DEBUG   ] summary = ERROR
2018-11-27 11:33:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643631  13 58449 41584605 A  ERRORERROR'
2018-11-27 11:33:04 [DEBUG   ] Got command RPT from MCS: ref #643633
2018-11-27 11:33:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:33:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643633  77 58449 41584800 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:33:05 [DEBUG   ] Got command RPT from MCS: ref #643634
2018-11-27 11:33:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:33:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643634  12 58449 41585001 A  ERRORB¢ff'
2018-11-27 11:33:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:33:18 [DEBUG   ] Got command BAM from MCS: ref #643640
2018-11-27 11:33:18 [INFO    ] Finished processing BAM request
2018-11-27 11:33:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:33:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643640   8 58449 41598029 A  ERROR'
2018-11-27 11:33:18 [DEBUG   ] Got command BAM from MCS: ref #643639
2018-11-27 11:33:18 [INFO    ] Finished processing BAM request
2018-11-27 11:33:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:33:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643639   8 58449 41598033 A  ERROR'
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:33:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:33:18 [DEBUG   ] spawn_process: at 2018-11-27 11:33:18 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:33:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:33:41 [DEBUG   ] Got command BAM from MCS: ref #643641
2018-11-27 11:33:41 [INFO    ] Finished processing BAM request
2018-11-27 11:33:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:33:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643641   8 58449 41621029 A  ERROR'
2018-11-27 11:33:41 [DEBUG   ] Got command BAM from MCS: ref #643642
2018-11-27 11:33:41 [INFO    ] Finished processing BAM request
2018-11-27 11:33:41 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:33:41 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643642   8 58449 41621033 A  ERROR'
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:33:41 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:33:41 [DEBUG   ] spawn_process: at 2018-11-27 11:33:41 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:33:54 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:33:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:33:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:33:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:33:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:33:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:33:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:33:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:33:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:33:58 [DEBUG   ] Finished updating board flags in 1.282 seconds
2018-11-27 11:34:04 [DEBUG   ] Got command BAM from MCS: ref #643644
2018-11-27 11:34:04 [INFO    ] Finished processing BAM request
2018-11-27 11:34:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:34:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643644   8 58449 41644029 A  ERROR'
2018-11-27 11:34:04 [DEBUG   ] Got command BAM from MCS: ref #643643
2018-11-27 11:34:04 [INFO    ] Finished processing BAM request
2018-11-27 11:34:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:34:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643643   8 58449 41644033 A  ERROR'
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:34:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:34:04 [DEBUG   ] spawn_process: at 2018-11-27 11:34:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:34:14 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 11:34:27 [DEBUG   ] Got command BAM from MCS: ref #643650
2018-11-27 11:34:27 [INFO    ] Finished processing BAM request
2018-11-27 11:34:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:34:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643650   8 58449 41667025 A  ERROR'
2018-11-27 11:34:27 [DEBUG   ] Got command BAM from MCS: ref #643649
2018-11-27 11:34:27 [INFO    ] Finished processing BAM request
2018-11-27 11:34:27 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:34:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643649   8 58449 41667029 A  ERROR'
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:34:27 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:34:27 [DEBUG   ] spawn_process: at 2018-11-27 11:34:27 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:34:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:34:50 [DEBUG   ] Got command BAM from MCS: ref #643651
2018-11-27 11:34:50 [INFO    ] Finished processing BAM request
2018-11-27 11:34:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:34:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643651   8 58449 41690025 A  ERROR'
2018-11-27 11:34:50 [DEBUG   ] Got command BAM from MCS: ref #643652
2018-11-27 11:34:50 [INFO    ] Finished processing BAM request
2018-11-27 11:34:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:34:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643652   8 58449 41690029 A  ERROR'
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:34:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:34:50 [DEBUG   ] spawn_process: at 2018-11-27 11:34:50 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:34:54 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:34:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:34:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 11:34:59 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 11:34:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 11:34:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:34:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:34:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:34:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:34:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:34:59 [DEBUG   ] Finished updating board flags in 1.756 seconds
2018-11-27 11:35:13 [DEBUG   ] Got command BAM from MCS: ref #643654
2018-11-27 11:35:13 [INFO    ] Finished processing BAM request
2018-11-27 11:35:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:35:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643654   8 58449 41713029 A  ERROR'
2018-11-27 11:35:13 [DEBUG   ] Got command BAM from MCS: ref #643653
2018-11-27 11:35:13 [INFO    ] Finished processing BAM request
2018-11-27 11:35:13 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:35:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643653   8 58449 41713033 A  ERROR'
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:35:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:35:13 [DEBUG   ] spawn_process: at 2018-11-27 11:35:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:35:14 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 11:35:34 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:35:36 [DEBUG   ] Got command BAM from MCS: ref #643655
2018-11-27 11:35:36 [INFO    ] Finished processing BAM request
2018-11-27 11:35:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:35:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643655   8 58449 41736033 A  ERROR'
2018-11-27 11:35:36 [DEBUG   ] Got command BAM from MCS: ref #643656
2018-11-27 11:35:36 [INFO    ] Finished processing BAM request
2018-11-27 11:35:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:35:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643656   8 58449 41736037 A  ERROR'
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:35:36 [DEBUG   ] spawn_process: at 2018-11-27 11:35:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:35:49 [DEBUG   ] Got command RPT from MCS: ref #643657
2018-11-27 11:35:49 [DEBUG   ] summary = ERROR
2018-11-27 11:35:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643657  13 58449 41749021 A  ERRORERROR'
2018-11-27 11:35:49 [DEBUG   ] Got command RPT from MCS: ref #643658
2018-11-27 11:35:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:35:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643658  77 58449 41749022 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:35:54 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:35:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 11:35:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 11:35:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:35:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:35:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:35:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:35:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:35:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:35:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:35:58 [DEBUG   ] Finished updating board flags in 1.015 seconds
2018-11-27 11:35:59 [DEBUG   ] Got command BAM from MCS: ref #643674
2018-11-27 11:35:59 [INFO    ] Finished processing BAM request
2018-11-27 11:35:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:35:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643674   8 58449 41759033 A  ERROR'
2018-11-27 11:35:59 [DEBUG   ] Got command BAM from MCS: ref #643675
2018-11-27 11:35:59 [INFO    ] Finished processing BAM request
2018-11-27 11:35:59 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:35:59 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643675   8 58449 41759037 A  ERROR'
2018-11-27 11:35:59 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:35:59 [DEBUG   ] spawn_process: at 2018-11-27 11:35:59 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:36:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:36:00 [DEBUG   ] spawn_process: at 2018-11-27 11:36:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:36:14 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2018-11-27 11:36:22 [DEBUG   ] Got command BAM from MCS: ref #643677
2018-11-27 11:36:22 [INFO    ] Finished processing BAM request
2018-11-27 11:36:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:36:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643677   8 58449 41782033 A  ERROR'
2018-11-27 11:36:22 [DEBUG   ] Got command BAM from MCS: ref #643676
2018-11-27 11:36:22 [INFO    ] Finished processing BAM request
2018-11-27 11:36:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:36:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643676   8 58449 41782037 A  ERROR'
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:36:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:36:22 [DEBUG   ] spawn_process: at 2018-11-27 11:36:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:36:34 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:36:45 [DEBUG   ] Got command BAM from MCS: ref #643678
2018-11-27 11:36:45 [INFO    ] Finished processing BAM request
2018-11-27 11:36:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:36:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643678   8 58449 41805033 A  ERROR'
2018-11-27 11:36:45 [DEBUG   ] Got command BAM from MCS: ref #643679
2018-11-27 11:36:45 [INFO    ] Finished processing BAM request
2018-11-27 11:36:45 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:36:45 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643679   8 58449 41805037 A  ERROR'
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 11:36:45 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 11:36:45 [DEBUG   ] spawn_process: at 2018-11-27 11:36:45 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 11:36:54 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:36:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:36:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:36:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:36:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:36:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:36:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:36:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:36:58 [DEBUG   ] Finished updating board flags in 1.012 seconds
2018-11-27 11:37:08 [DEBUG   ] Got command BAM from MCS: ref #643681
2018-11-27 11:37:08 [INFO    ] Finished processing BAM request
2018-11-27 11:37:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:37:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643681   8 58449 41828029 A  ERROR'
2018-11-27 11:37:08 [DEBUG   ] Got command BAM from MCS: ref #643680
2018-11-27 11:37:08 [INFO    ] Finished processing BAM request
2018-11-27 11:37:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:37:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643680   8 58449 41828033 A  ERROR'
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:37:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:37:08 [DEBUG   ] spawn_process: at 2018-11-27 11:37:08 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:37:14 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:37:31 [DEBUG   ] Got command BAM from MCS: ref #643687
2018-11-27 11:37:31 [INFO    ] Finished processing BAM request
2018-11-27 11:37:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:37:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643687   8 58449 41851029 A  ERROR'
2018-11-27 11:37:31 [DEBUG   ] Got command BAM from MCS: ref #643686
2018-11-27 11:37:31 [INFO    ] Finished processing BAM request
2018-11-27 11:37:31 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:37:31 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643686   8 58449 41851033 A  ERROR'
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 11:37:31 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 11:37:31 [DEBUG   ] spawn_process: at 2018-11-27 11:37:31 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 11:37:34 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:37:54 [DEBUG   ] Got command BAM from MCS: ref #643689
2018-11-27 11:37:54 [INFO    ] Finished processing BAM request
2018-11-27 11:37:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:37:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643689   8 58449 41874033 A  ERROR'
2018-11-27 11:37:54 [DEBUG   ] Got command BAM from MCS: ref #643688
2018-11-27 11:37:54 [INFO    ] Finished processing BAM request
2018-11-27 11:37:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 11:37:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   643688   8 58449 41874037 A  ERROR'
2018-11-27 11:37:54 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 11:37:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 11:37:54 [DEBUG   ] spawn_process: at 2018-11-27 11:37:54 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 11:37:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 11:37:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:37:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 11:37:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:37:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:37:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:37:58 [WARNING ] (13, 1, 'bot')
2018-11-27 11:37:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:37:58 [DEBUG   ] Finished updating board flags in 0.985 seconds
2018-11-27 11:38:02 [DEBUG   ] Got command STP from MCS: ref #643690
2018-11-27 11:38:02 [INFO    ] Finished processing BAM request
2018-11-27 11:38:02 [INFO    ] Shutdown the adder chain on beam #3
2018-11-27 11:38:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   643690   8 58449 41882025 A  ERROR'
2018-11-27 11:38:02 [DEBUG   ] Got command STP from MCS: ref #643691
2018-11-27 11:38:02 [INFO    ] Finished processing BAM request
2018-11-27 11:38:02 [INFO    ] Shutdown the adder chain on beam #2
2018-11-27 11:38:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   643691   8 58449 41882029 A  ERROR'
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 11:38:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 11:38:02 [DEBUG   ] spawn_process: at 2018-11-27 11:38:02 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 11:38:04 [DEBUG   ] Got command RPT from MCS: ref #643692
2018-11-27 11:38:04 [DEBUG   ] summary = ERROR
2018-11-27 11:38:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643692  13 58449 41884285 A  ERRORERROR'
2018-11-27 11:38:04 [DEBUG   ] Got command RPT from MCS: ref #643693
2018-11-27 11:38:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:38:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643693  77 58449 41884490 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:38:04 [DEBUG   ] Got command RPT from MCS: ref #643694
2018-11-27 11:38:04 [DEBUG   ] summary = ERROR
2018-11-27 11:38:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643694  13 58449 41884681 A  ERRORERROR'
2018-11-27 11:38:04 [DEBUG   ] Got command RPT from MCS: ref #643695
2018-11-27 11:38:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:38:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643695  77 58449 41884885 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:38:05 [DEBUG   ] Got command RPT from MCS: ref #643696
2018-11-27 11:38:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:38:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643696  12 58449 41885089 A  ERRORB¢ff'
2018-11-27 11:38:14 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:38:34 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:38:54 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 11:38:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:38:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 11:38:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:38:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:38:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:38:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:38:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:38:59 [DEBUG   ] Finished updating board flags in 1.428 seconds
2018-11-27 11:39:14 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:39:34 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 11:39:54 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 11:39:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:39:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:39:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:39:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:39:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:39:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:39:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:39:59 [DEBUG   ] Finished updating board flags in 1.338 seconds
2018-11-27 11:40:14 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2018-11-27 11:40:35 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 11:40:49 [DEBUG   ] Got command RPT from MCS: ref #643701
2018-11-27 11:40:49 [DEBUG   ] summary = ERROR
2018-11-27 11:40:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643701  13 58449 42049013 A  ERRORERROR'
2018-11-27 11:40:49 [DEBUG   ] Got command RPT from MCS: ref #643702
2018-11-27 11:40:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:40:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643702  77 58449 42049014 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:40:55 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:40:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:40:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:40:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:40:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:40:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:40:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:40:59 [DEBUG   ] Finished updating board flags in 1.221 seconds
2018-11-27 11:41:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:41:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:41:55 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:41:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 11:41:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:41:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:41:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:41:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:41:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:41:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:41:59 [DEBUG   ] Finished updating board flags in 1.181 seconds
2018-11-27 11:42:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:42:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:42:55 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:42:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:42:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:42:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 11:42:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:42:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:42:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:42:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:42:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:42:59 [DEBUG   ] Finished updating board flags in 1.007 seconds
2018-11-27 11:43:04 [DEBUG   ] Got command RPT from MCS: ref #643720
2018-11-27 11:43:04 [DEBUG   ] summary = ERROR
2018-11-27 11:43:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643720  13 58449 42184365 A  ERRORERROR'
2018-11-27 11:43:04 [DEBUG   ] Got command RPT from MCS: ref #643721
2018-11-27 11:43:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:43:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643721  77 58449 42184569 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:43:04 [DEBUG   ] Got command RPT from MCS: ref #643722
2018-11-27 11:43:04 [DEBUG   ] summary = ERROR
2018-11-27 11:43:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643722  13 58449 42184773 A  ERRORERROR'
2018-11-27 11:43:04 [DEBUG   ] Got command RPT from MCS: ref #643724
2018-11-27 11:43:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:43:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643724  77 58449 42184977 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:43:05 [DEBUG   ] Got command RPT from MCS: ref #643726
2018-11-27 11:43:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:43:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643726  12 58449 42185169 A  ERRORB™™š'
2018-11-27 11:43:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:43:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:43:55 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:43:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:43:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:43:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-27 11:43:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:43:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:43:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:43:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:43:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:43:59 [DEBUG   ] Finished updating board flags in 1.007 seconds
2018-11-27 11:44:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:44:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:44:55 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 11:44:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:44:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:44:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:44:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:44:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:44:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:44:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:44:59 [DEBUG   ] Finished updating board flags in 1.070 seconds
2018-11-27 11:45:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:45:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:45:49 [DEBUG   ] Got command RPT from MCS: ref #643734
2018-11-27 11:45:49 [DEBUG   ] summary = ERROR
2018-11-27 11:45:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643734  13 58449 42349017 A  ERRORERROR'
2018-11-27 11:45:49 [DEBUG   ] Got command RPT from MCS: ref #643735
2018-11-27 11:45:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:45:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643735  77 58449 42349018 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:45:55 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 11:45:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:45:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:45:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:45:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:45:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:45:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:45:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:45:59 [DEBUG   ] Finished updating board flags in 1.483 seconds
2018-11-27 11:46:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:46:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:46:55 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:46:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:46:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:46:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:46:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:46:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:46:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:46:59 [DEBUG   ] Finished updating board flags in 1.291 seconds
2018-11-27 11:47:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:47:35 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 11:47:55 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:47:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:47:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:47:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:47:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:47:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:47:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:47:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:47:59 [DEBUG   ] Finished updating board flags in 1.236 seconds
2018-11-27 11:48:04 [DEBUG   ] Got command RPT from MCS: ref #643755
2018-11-27 11:48:04 [DEBUG   ] summary = ERROR
2018-11-27 11:48:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643755  13 58449 42484457 A  ERRORERROR'
2018-11-27 11:48:04 [DEBUG   ] Got command RPT from MCS: ref #643756
2018-11-27 11:48:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:48:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643756  77 58449 42484661 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:48:04 [DEBUG   ] Got command RPT from MCS: ref #643757
2018-11-27 11:48:04 [DEBUG   ] summary = ERROR
2018-11-27 11:48:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643757  13 58449 42484853 A  ERRORERROR'
2018-11-27 11:48:05 [DEBUG   ] Got command RPT from MCS: ref #643758
2018-11-27 11:48:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:48:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643758  77 58449 42485057 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:48:05 [DEBUG   ] Got command RPT from MCS: ref #643759
2018-11-27 11:48:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:48:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643759  12 58449 42485261 A  ERRORB˜™š'
2018-11-27 11:48:15 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 11:48:35 [DEBUG   ] Finished updating ant stats in 0.962 seconds
2018-11-27 11:48:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:48:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:48:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:48:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:48:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:48:59 [WARNING ] (13, 1, 'bot')
2018-11-27 11:48:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:48:59 [DEBUG   ] Finished updating board flags in 1.163 seconds
2018-11-27 11:49:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:49:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:49:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:49:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:50:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:50:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:50:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:50:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:50:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:50:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:50:00 [DEBUG   ] Finished updating board flags in 1.569 seconds
2018-11-27 11:50:15 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 11:50:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:50:49 [DEBUG   ] Got command RPT from MCS: ref #643764
2018-11-27 11:50:49 [DEBUG   ] summary = ERROR
2018-11-27 11:50:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643764  13 58449 42649017 A  ERRORERROR'
2018-11-27 11:50:49 [DEBUG   ] Got command RPT from MCS: ref #643765
2018-11-27 11:50:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:50:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643765  77 58449 42649018 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:50:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:50:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:51:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:51:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:51:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:51:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:51:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:51:00 [DEBUG   ] Finished updating board flags in 1.390 seconds
2018-11-27 11:51:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:51:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:51:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:51:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:51:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:52:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:52:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:52:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:52:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:52:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:52:00 [DEBUG   ] Finished updating board flags in 1.221 seconds
2018-11-27 11:52:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:52:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 11:52:55 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2018-11-27 11:52:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 11:52:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:53:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:53:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:53:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:53:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:53:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:53:00 [DEBUG   ] Finished updating board flags in 1.202 seconds
2018-11-27 11:53:04 [DEBUG   ] Got command RPT from MCS: ref #643787
2018-11-27 11:53:04 [DEBUG   ] summary = ERROR
2018-11-27 11:53:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643787  13 58449 42784541 A  ERRORERROR'
2018-11-27 11:53:04 [DEBUG   ] Got command RPT from MCS: ref #643788
2018-11-27 11:53:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:53:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643788  77 58449 42784745 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:53:04 [DEBUG   ] Got command RPT from MCS: ref #643789
2018-11-27 11:53:04 [DEBUG   ] summary = ERROR
2018-11-27 11:53:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643789  13 58449 42784937 A  ERRORERROR'
2018-11-27 11:53:05 [DEBUG   ] Got command RPT from MCS: ref #643791
2018-11-27 11:53:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:53:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643791  77 58449 42785141 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:53:05 [DEBUG   ] Got command RPT from MCS: ref #643792
2018-11-27 11:53:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:53:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643792  12 58449 42785345 A  ERRORB˜™š'
2018-11-27 11:53:15 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 11:53:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:53:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:53:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:54:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 11:54:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:54:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:54:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:54:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:54:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:54:00 [DEBUG   ] Finished updating board flags in 1.214 seconds
2018-11-27 11:54:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:54:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:54:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:54:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:55:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:55:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:55:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:55:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:55:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:55:00 [DEBUG   ] Finished updating board flags in 1.238 seconds
2018-11-27 11:55:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:55:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:55:49 [DEBUG   ] Got command RPT from MCS: ref #643801
2018-11-27 11:55:49 [DEBUG   ] summary = ERROR
2018-11-27 11:55:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643801  13 58449 42949017 A  ERRORERROR'
2018-11-27 11:55:49 [DEBUG   ] Got command RPT from MCS: ref #643802
2018-11-27 11:55:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:55:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643802  77 58449 42949018 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:55:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:55:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:56:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:56:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 11:56:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:56:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:56:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:56:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:56:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:56:00 [DEBUG   ] Finished updating board flags in 1.138 seconds
2018-11-27 11:56:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:56:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:56:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:56:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa16, chip 1
2018-11-27 11:56:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:57:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:57:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:57:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:57:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:57:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:57:00 [DEBUG   ] Finished updating board flags in 1.034 seconds
2018-11-27 11:57:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:57:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 11:57:56 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 11:58:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:58:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 11:58:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 11:58:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:58:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:58:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:58:00 [WARNING ] (13, 1, 'bot')
2018-11-27 11:58:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:58:00 [DEBUG   ] Finished updating board flags in 1.472 seconds
2018-11-27 11:58:04 [DEBUG   ] Got command RPT from MCS: ref #643818
2018-11-27 11:58:04 [DEBUG   ] summary = ERROR
2018-11-27 11:58:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643818  13 58449 43084629 A  ERRORERROR'
2018-11-27 11:58:04 [DEBUG   ] Got command RPT from MCS: ref #643819
2018-11-27 11:58:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:58:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643819  77 58449 43084821 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:58:05 [DEBUG   ] Got command RPT from MCS: ref #643820
2018-11-27 11:58:05 [DEBUG   ] summary = ERROR
2018-11-27 11:58:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643820  13 58449 43085025 A  ERRORERROR'
2018-11-27 11:58:05 [DEBUG   ] Got command RPT from MCS: ref #643821
2018-11-27 11:58:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 11:58:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643821  77 58449 43085229 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 11:58:05 [DEBUG   ] Got command RPT from MCS: ref #643822
2018-11-27 11:58:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 11:58:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643822  12 58449 43085433 A  ERRORB˜™š'
2018-11-27 11:58:16 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:58:36 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:58:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 11:59:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 11:59:01 [WARNING ] (13, 1, 'bot')
2018-11-27 11:59:01 [WARNING ] (13, 1, 'bot')
2018-11-27 11:59:01 [WARNING ] (13, 1, 'bot')
2018-11-27 11:59:01 [WARNING ] (13, 1, 'bot')
2018-11-27 11:59:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 11:59:01 [DEBUG   ] Finished updating board flags in 1.940 seconds
2018-11-27 11:59:16 [DEBUG   ] Finished updating ant stats in 0.994 seconds
2018-11-27 11:59:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 11:59:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:00:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:00:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:00:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:00:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:00:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:00:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:00:00 [DEBUG   ] Finished updating board flags in 1.204 seconds
2018-11-27 12:00:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:00:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:00:49 [DEBUG   ] Got command RPT from MCS: ref #643827
2018-11-27 12:00:49 [DEBUG   ] summary = ERROR
2018-11-27 12:00:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643827  13 58449 43249025 A  ERRORERROR'
2018-11-27 12:00:49 [DEBUG   ] Got command RPT from MCS: ref #643828
2018-11-27 12:00:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:00:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643828  77 58449 43249026 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:00:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:01:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:01:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 12:01:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:01:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:01:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:01:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:01:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:01:01 [DEBUG   ] Finished updating board flags in 1.769 seconds
2018-11-27 12:01:16 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 12:01:36 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 12:01:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:02:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:02:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:02:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:02:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:02:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:02:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:02:00 [DEBUG   ] Finished updating board flags in 1.022 seconds
2018-11-27 12:02:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:02:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:02:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:03:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:03:00 [DEBUG   ] Got command RPT from MCS: ref #643850
2018-11-27 12:03:00 [DEBUG   ] summary = ERROR
2018-11-27 12:03:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643850  13 58449 43380925 A  ERRORERROR'
2018-11-27 12:03:00 [DEBUG   ] Got command RPT from MCS: ref #643851
2018-11-27 12:03:00 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:03:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643851  77 58449 43380926 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:03:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:03:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:03:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:03:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:03:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:03:00 [DEBUG   ] Finished updating board flags in 1.080 seconds
2018-11-27 12:03:04 [DEBUG   ] Got command RPT from MCS: ref #643852
2018-11-27 12:03:04 [DEBUG   ] summary = ERROR
2018-11-27 12:03:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643852  13 58449 43384705 A  ERRORERROR'
2018-11-27 12:03:04 [DEBUG   ] Got command RPT from MCS: ref #643853
2018-11-27 12:03:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:03:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643853  77 58449 43384909 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:03:05 [DEBUG   ] Got command RPT from MCS: ref #643854
2018-11-27 12:03:05 [DEBUG   ] summary = ERROR
2018-11-27 12:03:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643854  13 58449 43385113 A  ERRORERROR'
2018-11-27 12:03:05 [DEBUG   ] Got command RPT from MCS: ref #643856
2018-11-27 12:03:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:03:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643856  77 58449 43385317 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:03:05 [DEBUG   ] Got command RPT from MCS: ref #643857
2018-11-27 12:03:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:03:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643857  12 58449 43385509 A  ERRORB˜™š'
2018-11-27 12:03:05 [DEBUG   ] Got command INI from MCS: ref #643861
2018-11-27 12:03:05 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2018-11-27 12:03:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   643861   8 58449 43385933 ABOOTING'
2018-11-27 12:03:26 [INFO    ] -------------------
2018-11-27 12:03:26 [INFO    ]  LWA board status
2018-11-27 12:03:26 [INFO    ] -------------------
2018-11-27 12:03:26 [INFO    ] DP 1: 2 lwa27
2018-11-27 12:03:26 [INFO    ] DP 1: 3 lwa24
2018-11-27 12:03:26 [INFO    ] DP 1: 4 lwa23
2018-11-27 12:03:26 [INFO    ] DP 1: 5 lwa18
2018-11-27 12:03:26 [INFO    ] DP 1: 6 lwa10
2018-11-27 12:03:26 [INFO    ] DP 1: 7 lwa32
2018-11-27 12:03:26 [INFO    ] DP 1: 8 lwa11
2018-11-27 12:03:26 [INFO    ] DP 1: 9 lwa30
2018-11-27 12:03:26 [INFO    ] DP 1: 10 lwa12
2018-11-27 12:03:26 [INFO    ] DP 1: 11 lwa8
2018-11-27 12:03:26 [INFO    ] DP 1: 12 lwa13
2018-11-27 12:03:26 [INFO    ] DP 1: 13 lwa26
2018-11-27 12:03:26 [INFO    ] DP 1: 14 lwa14
2018-11-27 12:03:26 [INFO    ] DP 1: 16 lwa22
2018-11-27 12:03:26 [INFO    ] DP 1: 17 lwa20
2018-11-27 12:03:26 [INFO    ] DP 1: 18 lwa1
2018-11-27 12:03:26 [INFO    ] DP 1: 19 lwa5
2018-11-27 12:03:26 [INFO    ] DP 1: 20 lwa25
2018-11-27 12:03:26 [INFO    ] DP 1: 21 lwa35
2018-11-27 12:03:26 [INFO    ] DP 1: 22 lwa19
2018-11-27 12:03:26 [INFO    ] DP 1: 23 lwa21
2018-11-27 12:03:26 [INFO    ] DP 1: 24 lwa6
2018-11-27 12:03:26 [INFO    ] DP 1: 25 lwa7
2018-11-27 12:03:26 [INFO    ] DP 1: 26 lwa33
2018-11-27 12:03:26 [INFO    ] DP 1: 27 lwa15
2018-11-27 12:03:26 [INFO    ] DP 1: 28 lwa28
2018-11-27 12:03:26 [INFO    ] DP 2: 1 lwa16
2018-11-27 12:03:26 [INFO    ] DP 2: 15 lwa34
2018-11-27 12:03:26 [INFO    ] Total of 28 boards
2018-11-27 12:03:26 [INFO    ] Total of 260 stands
2018-11-27 12:03:26 [INFO    ] -------------------
2018-11-27 12:03:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26419
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26420
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26421
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26422
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26423
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26424
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26425
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26426
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26427
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26428
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26429
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26430
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26431
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26432
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26433
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26434
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26435
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26436
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26437
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26438
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26439
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26440
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26441
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26442
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26443
2018-11-27 12:03:33 [DEBUG   ] 2018-11-27 12:03:33 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 26444
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26419 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26420 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26421 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26422 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26423 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26424 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26425 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26426 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26427 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26428 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26429 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26430 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26431 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26432 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26433 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26434 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26435 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26436 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26437 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26438 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26439 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26440 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26441 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26442 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26443 exited with status 0
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 26444 exited with status 0
2018-11-27 12:03:45 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] spawn_process: at 2018-11-27 12:03:45 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 26471
2018-11-27 12:03:45 [DEBUG   ] 2018-11-27 12:03:45 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 26472
2018-11-27 12:03:51 [DEBUG   ] 2018-11-27 12:03:51 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 26471 exited with status 0
2018-11-27 12:03:51 [DEBUG   ] 2018-11-27 12:03:51 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 26472 exited with status 0
2018-11-27 12:03:51 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2018-11-27 12:03:51 [DEBUG   ] spawn_process: at 2018-11-27 12:03:51 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2018-11-27 12:03:51 [DEBUG   ] spawn_process: at 2018-11-27 12:03:51 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2018-11-27 12:03:51 [INFO    ] Setting Beam 1 xbars
2018-11-27 12:03:51 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2018-11-27 12:03:51 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2018-11-27 12:03:51 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2018-11-27 12:03:51 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2018-11-27 12:03:51 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2018-11-27 12:03:52 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2018-11-27 12:03:52 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2018-11-27 12:03:52 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2018-11-27 12:03:52 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2018-11-27 12:03:52 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2018-11-27 12:03:52 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2018-11-27 12:03:52 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2018-11-27 12:03:52 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2018-11-27 12:03:52 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2018-11-27 12:03:53 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2018-11-27 12:03:53 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2018-11-27 12:03:53 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2018-11-27 12:03:53 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2018-11-27 12:03:53 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2018-11-27 12:03:53 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2018-11-27 12:03:53 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2018-11-27 12:03:53 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2018-11-27 12:03:53 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2018-11-27 12:03:53 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2018-11-27 12:03:54 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2018-11-27 12:03:54 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2018-11-27 12:03:54 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2018-11-27 12:03:54 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2018-11-27 12:03:54 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2018-11-27 12:03:54 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2018-11-27 12:03:54 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2018-11-27 12:03:54 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2018-11-27 12:03:54 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2018-11-27 12:03:55 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2018-11-27 12:03:55 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2018-11-27 12:03:55 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2018-11-27 12:03:55 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2018-11-27 12:03:55 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2018-11-27 12:03:55 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2018-11-27 12:03:55 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2018-11-27 12:03:55 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2018-11-27 12:03:55 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2018-11-27 12:03:56 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2018-11-27 12:03:56 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2018-11-27 12:03:56 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2018-11-27 12:03:56 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2018-11-27 12:03:56 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2018-11-27 12:03:56 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2018-11-27 12:03:56 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2018-11-27 12:03:56 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2018-11-27 12:03:56 [INFO    ] Setting Beam 2 xbars
2018-11-27 12:03:56 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2018-11-27 12:03:57 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2018-11-27 12:03:57 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2018-11-27 12:03:57 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2018-11-27 12:03:57 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2018-11-27 12:03:57 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2018-11-27 12:03:57 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2018-11-27 12:03:57 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2018-11-27 12:03:57 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2018-11-27 12:03:57 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2018-11-27 12:03:57 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2018-11-27 12:03:58 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2018-11-27 12:03:58 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2018-11-27 12:03:58 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2018-11-27 12:03:58 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2018-11-27 12:03:58 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2018-11-27 12:03:58 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2018-11-27 12:03:58 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2018-11-27 12:03:58 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2018-11-27 12:03:58 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2018-11-27 12:03:59 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2018-11-27 12:03:59 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2018-11-27 12:03:59 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2018-11-27 12:03:59 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2018-11-27 12:03:59 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2018-11-27 12:03:59 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2018-11-27 12:03:59 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2018-11-27 12:03:59 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2018-11-27 12:03:59 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2018-11-27 12:04:00 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2018-11-27 12:04:00 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2018-11-27 12:04:00 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2018-11-27 12:04:00 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2018-11-27 12:04:00 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2018-11-27 12:04:00 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2018-11-27 12:04:00 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2018-11-27 12:04:00 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2018-11-27 12:04:00 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2018-11-27 12:04:01 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2018-11-27 12:04:01 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2018-11-27 12:04:01 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2018-11-27 12:04:01 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2018-11-27 12:04:01 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2018-11-27 12:04:01 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2018-11-27 12:04:01 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2018-11-27 12:04:01 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2018-11-27 12:04:01 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2018-11-27 12:04:01 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2018-11-27 12:04:02 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2018-11-27 12:04:02 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2018-11-27 12:04:02 [INFO    ] Setting Beam 3 xbars
2018-11-27 12:04:02 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2018-11-27 12:04:02 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2018-11-27 12:04:02 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2018-11-27 12:04:02 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2018-11-27 12:04:02 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2018-11-27 12:04:02 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2018-11-27 12:04:02 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2018-11-27 12:04:03 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2018-11-27 12:04:03 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2018-11-27 12:04:03 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2018-11-27 12:04:03 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2018-11-27 12:04:03 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2018-11-27 12:04:03 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2018-11-27 12:04:03 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2018-11-27 12:04:03 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2018-11-27 12:04:03 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2018-11-27 12:04:04 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2018-11-27 12:04:04 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2018-11-27 12:04:04 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2018-11-27 12:04:04 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2018-11-27 12:04:04 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2018-11-27 12:04:04 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2018-11-27 12:04:04 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2018-11-27 12:04:04 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2018-11-27 12:04:04 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2018-11-27 12:04:05 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2018-11-27 12:04:05 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2018-11-27 12:04:05 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2018-11-27 12:04:05 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2018-11-27 12:04:05 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2018-11-27 12:04:05 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2018-11-27 12:04:05 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2018-11-27 12:04:05 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2018-11-27 12:04:05 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2018-11-27 12:04:06 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2018-11-27 12:04:06 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2018-11-27 12:04:06 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2018-11-27 12:04:06 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2018-11-27 12:04:06 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2018-11-27 12:04:06 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2018-11-27 12:04:06 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2018-11-27 12:04:06 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2018-11-27 12:04:06 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2018-11-27 12:04:06 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2018-11-27 12:04:07 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2018-11-27 12:04:07 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2018-11-27 12:04:07 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2018-11-27 12:04:07 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2018-11-27 12:04:07 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2018-11-27 12:04:07 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2018-11-27 12:04:07 [INFO    ] Setting Beam 4 xbars
2018-11-27 12:04:07 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2018-11-27 12:04:07 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 12:04:07 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 12:04:08 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 12:04:08 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 12:04:08 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2018-11-27 12:04:08 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2018-11-27 12:04:08 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2018-11-27 12:04:08 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2018-11-27 12:04:08 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2018-11-27 12:04:08 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2018-11-27 12:04:08 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2018-11-27 12:04:09 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2018-11-27 12:04:09 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2018-11-27 12:04:09 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2018-11-27 12:04:09 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2018-11-27 12:04:09 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2018-11-27 12:04:09 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2018-11-27 12:04:09 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2018-11-27 12:04:09 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2018-11-27 12:04:09 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2018-11-27 12:04:10 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2018-11-27 12:04:10 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2018-11-27 12:04:10 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2018-11-27 12:04:10 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2018-11-27 12:04:10 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2018-11-27 12:04:10 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2018-11-27 12:04:10 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2018-11-27 12:04:10 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2018-11-27 12:04:10 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2018-11-27 12:04:11 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2018-11-27 12:04:11 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2018-11-27 12:04:11 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2018-11-27 12:04:11 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2018-11-27 12:04:11 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2018-11-27 12:04:11 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2018-11-27 12:04:11 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2018-11-27 12:04:11 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2018-11-27 12:04:11 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2018-11-27 12:04:11 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2018-11-27 12:04:12 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2018-11-27 12:04:12 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2018-11-27 12:04:12 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2018-11-27 12:04:12 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2018-11-27 12:04:12 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2018-11-27 12:04:12 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2018-11-27 12:04:12 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2018-11-27 12:04:12 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2018-11-27 12:04:12 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2018-11-27 12:04:13 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2018-11-27 12:04:13 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2018-11-27 12:04:13 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2018-11-27 12:04:13 [INFO    ] All xbars set
2018-11-27 12:04:13 [INFO    ] Setting adder chain routing for all beams
2018-11-27 12:04:18 [INFO    ] Setting beam ids
2018-11-27 12:04:28 [INFO    ] Starting beam delay calibration
2018-11-27 12:04:40 [INFO    ] Beam 1 latency is 5726
2018-11-27 12:04:40 [INFO    ] Beam 2 latency is 6705
2018-11-27 12:04:40 [INFO    ] Beam 3 latency is 6694
2018-11-27 12:04:40 [INFO    ] Beam 4 latency is 5725
2018-11-27 12:04:40 [INFO    ] Stop calibration
2018-11-27 12:04:48 [ERROR   ] Beam delay calibration failed
2018-11-27 12:04:48 [CRITICAL] Calibration failed
2018-11-27 12:04:48 [DEBUG   ] spawn_process: at 2018-11-27 12:04:48 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2018-11-27 12:04:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2018-11-27 12:04:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2018-11-27 12:04:48 [DEBUG   ] spawn_process: at 2018-11-27 12:04:48 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2018-11-27 12:04:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2018-11-27 12:04:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2018-11-27 12:04:48 [DEBUG   ] spawn_process: at 2018-11-27 12:04:48 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2018-11-27 12:04:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2018-11-27 12:04:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2018-11-27 12:04:48 [DEBUG   ] spawn_process: at 2018-11-27 12:04:48 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2018-11-27 12:04:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2018-11-27 12:04:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2018-11-27 12:04:48 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 12:04:49 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 274 ms (1543320289.276 s) with PID 28050 and time 1543320290
2018-11-27 12:04:49 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 12:04:49 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 364 ms (1543320289.365 s) with PID 28053 and time 1543320290
2018-11-27 12:04:50 [DEBUG   ] Finished updating ant stats in 0.881 seconds
2018-11-27 12:04:50 [INFO    ] Started antennas statistics background thread
2018-11-27 12:04:50 [INFO    ] Started board flags background thread
2018-11-27 12:04:50 [INFO    ] Started BAM background thread
2018-11-27 12:04:50 [INFO    ] Finished the INI process (ref. #643861) in 104.556 s
2018-11-27 12:04:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:04:51 [WARNING ] (13, 1, 'bot')
2018-11-27 12:04:51 [WARNING ] (13, 1, 'bot')
2018-11-27 12:04:51 [WARNING ] (13, 1, 'bot')
2018-11-27 12:04:51 [WARNING ] (13, 1, 'bot')
2018-11-27 12:04:51 [CRITICAL] MGT underflow detected on beam(s) 1, 2, 3, 4
2018-11-27 12:04:51 [DEBUG   ] Finished updating board flags in 1.063 seconds
2018-11-27 12:05:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:05:30 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:05:49 [DEBUG   ] Got command RPT from MCS: ref #643867
2018-11-27 12:05:49 [DEBUG   ] summary = ERROR
2018-11-27 12:05:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643867  13 58449 43549027 A  ERRORERROR'
2018-11-27 12:05:49 [DEBUG   ] Got command RPT from MCS: ref #643868
2018-11-27 12:05:49 [DEBUG   ] info = T_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:05:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643868  98 58449 43549028 A  ERRORT_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:05:50 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:05:50 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:05:51 [WARNING ] (13, 1, 'bot')
2018-11-27 12:05:51 [WARNING ] (13, 1, 'bot')
2018-11-27 12:05:51 [WARNING ] (13, 1, 'bot')
2018-11-27 12:05:51 [WARNING ] (13, 1, 'bot')
2018-11-27 12:05:51 [CRITICAL] MGT underflow detected on beam(s) 1, 2, 3, 4
2018-11-27 12:05:51 [DEBUG   ] Finished updating board flags in 1.010 seconds
2018-11-27 12:06:05 [DEBUG   ] Got command RPT from MCS: ref #643884
2018-11-27 12:06:05 [DEBUG   ] summary = ERROR
2018-11-27 12:06:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643884  13 58449 43565954 A  ERRORERROR'
2018-11-27 12:06:05 [DEBUG   ] Got command RPT from MCS: ref #643885
2018-11-27 12:06:05 [DEBUG   ] info = T_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:06:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643885  98 58449 43565955 A  ERRORT_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:06:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:06:10 [DEBUG   ] Got command INI from MCS: ref #643886
2018-11-27 12:06:10 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2018-11-27 12:06:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   643886   8 58449 43570938 ABOOTING'
2018-11-27 12:06:32 [INFO    ] -------------------
2018-11-27 12:06:32 [INFO    ]  LWA board status
2018-11-27 12:06:32 [INFO    ] -------------------
2018-11-27 12:06:32 [INFO    ] DP 1: 2 lwa27
2018-11-27 12:06:32 [INFO    ] DP 1: 3 lwa24
2018-11-27 12:06:32 [INFO    ] DP 1: 4 lwa23
2018-11-27 12:06:32 [INFO    ] DP 1: 5 lwa18
2018-11-27 12:06:32 [INFO    ] DP 1: 6 lwa10
2018-11-27 12:06:32 [INFO    ] DP 1: 7 lwa32
2018-11-27 12:06:32 [INFO    ] DP 1: 8 lwa11
2018-11-27 12:06:32 [INFO    ] DP 1: 9 lwa30
2018-11-27 12:06:32 [INFO    ] DP 1: 10 lwa12
2018-11-27 12:06:32 [INFO    ] DP 1: 11 lwa8
2018-11-27 12:06:32 [INFO    ] DP 1: 12 lwa13
2018-11-27 12:06:32 [INFO    ] DP 1: 13 lwa26
2018-11-27 12:06:32 [INFO    ] DP 1: 14 lwa14
2018-11-27 12:06:32 [INFO    ] DP 1: 16 lwa22
2018-11-27 12:06:32 [INFO    ] DP 1: 17 lwa20
2018-11-27 12:06:32 [INFO    ] DP 1: 18 lwa1
2018-11-27 12:06:32 [INFO    ] DP 1: 19 lwa5
2018-11-27 12:06:32 [INFO    ] DP 1: 20 lwa25
2018-11-27 12:06:32 [INFO    ] DP 1: 21 lwa35
2018-11-27 12:06:32 [INFO    ] DP 1: 22 lwa19
2018-11-27 12:06:32 [INFO    ] DP 1: 23 lwa21
2018-11-27 12:06:32 [INFO    ] DP 1: 24 lwa6
2018-11-27 12:06:32 [INFO    ] DP 1: 25 lwa7
2018-11-27 12:06:32 [INFO    ] DP 1: 26 lwa33
2018-11-27 12:06:32 [INFO    ] DP 1: 27 lwa15
2018-11-27 12:06:32 [INFO    ] DP 1: 28 lwa28
2018-11-27 12:06:32 [INFO    ] DP 2: 1 lwa16
2018-11-27 12:06:32 [INFO    ] DP 2: 15 lwa34
2018-11-27 12:06:32 [INFO    ] Total of 28 boards
2018-11-27 12:06:32 [INFO    ] Total of 260 stands
2018-11-27 12:06:32 [INFO    ] -------------------
2018-11-27 12:06:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:34 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:35 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28609
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28610
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28611
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28612
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28613
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28614
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28615
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28616
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28617
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28618
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28619
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28620
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28621
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28622
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28623
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28624
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28625
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28626
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28627
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28628
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28629
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28630
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28631
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28632
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28633
2018-11-27 12:06:39 [DEBUG   ] 2018-11-27 12:06:39 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 28634
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28609 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28610 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28611 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28612 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28613 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28614 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28615 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28616 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28617 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28618 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28619 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28620 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28621 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28622 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28623 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28624 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28625 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28626 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28627 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28628 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28629 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28630 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28631 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28632 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28633 exited with status 0
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 28634 exited with status 0
2018-11-27 12:06:51 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] spawn_process: at 2018-11-27 12:06:51 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 28661
2018-11-27 12:06:51 [DEBUG   ] 2018-11-27 12:06:51 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 28662
2018-11-27 12:06:57 [DEBUG   ] 2018-11-27 12:06:57 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 28661 exited with status 0
2018-11-27 12:06:57 [DEBUG   ] 2018-11-27 12:06:57 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 28662 exited with status 0
2018-11-27 12:06:57 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2018-11-27 12:06:57 [DEBUG   ] spawn_process: at 2018-11-27 12:06:57 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2018-11-27 12:06:57 [DEBUG   ] spawn_process: at 2018-11-27 12:06:57 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2018-11-27 12:06:57 [INFO    ] Setting Beam 1 xbars
2018-11-27 12:06:57 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2018-11-27 12:06:57 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2018-11-27 12:06:57 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2018-11-27 12:06:57 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2018-11-27 12:06:57 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2018-11-27 12:06:57 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2018-11-27 12:06:57 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2018-11-27 12:06:57 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2018-11-27 12:06:57 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2018-11-27 12:06:58 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2018-11-27 12:06:58 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2018-11-27 12:06:58 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2018-11-27 12:06:58 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2018-11-27 12:06:58 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2018-11-27 12:06:58 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2018-11-27 12:06:58 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2018-11-27 12:06:58 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2018-11-27 12:06:58 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2018-11-27 12:06:59 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2018-11-27 12:06:59 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2018-11-27 12:06:59 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2018-11-27 12:06:59 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2018-11-27 12:06:59 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2018-11-27 12:06:59 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2018-11-27 12:06:59 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2018-11-27 12:06:59 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2018-11-27 12:06:59 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2018-11-27 12:07:00 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2018-11-27 12:07:00 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2018-11-27 12:07:00 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2018-11-27 12:07:00 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2018-11-27 12:07:00 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2018-11-27 12:07:00 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2018-11-27 12:07:00 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2018-11-27 12:07:00 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2018-11-27 12:07:00 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2018-11-27 12:07:01 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2018-11-27 12:07:01 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2018-11-27 12:07:01 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2018-11-27 12:07:01 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2018-11-27 12:07:01 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2018-11-27 12:07:01 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2018-11-27 12:07:01 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2018-11-27 12:07:01 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2018-11-27 12:07:01 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2018-11-27 12:07:01 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2018-11-27 12:07:02 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2018-11-27 12:07:02 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2018-11-27 12:07:02 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2018-11-27 12:07:02 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2018-11-27 12:07:02 [INFO    ] Setting Beam 2 xbars
2018-11-27 12:07:02 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2018-11-27 12:07:02 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2018-11-27 12:07:02 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2018-11-27 12:07:02 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2018-11-27 12:07:02 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2018-11-27 12:07:03 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2018-11-27 12:07:03 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2018-11-27 12:07:03 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2018-11-27 12:07:03 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2018-11-27 12:07:03 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2018-11-27 12:07:03 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2018-11-27 12:07:03 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2018-11-27 12:07:03 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2018-11-27 12:07:03 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2018-11-27 12:07:04 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2018-11-27 12:07:04 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2018-11-27 12:07:04 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2018-11-27 12:07:04 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2018-11-27 12:07:04 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2018-11-27 12:07:04 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2018-11-27 12:07:04 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2018-11-27 12:07:04 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2018-11-27 12:07:04 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2018-11-27 12:07:04 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2018-11-27 12:07:05 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2018-11-27 12:07:05 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2018-11-27 12:07:05 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2018-11-27 12:07:05 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2018-11-27 12:07:05 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2018-11-27 12:07:05 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2018-11-27 12:07:05 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2018-11-27 12:07:05 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2018-11-27 12:07:05 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2018-11-27 12:07:06 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2018-11-27 12:07:06 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2018-11-27 12:07:06 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2018-11-27 12:07:06 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2018-11-27 12:07:06 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2018-11-27 12:07:06 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2018-11-27 12:07:06 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2018-11-27 12:07:06 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2018-11-27 12:07:06 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2018-11-27 12:07:07 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2018-11-27 12:07:07 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2018-11-27 12:07:07 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2018-11-27 12:07:07 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2018-11-27 12:07:07 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2018-11-27 12:07:07 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2018-11-27 12:07:07 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2018-11-27 12:07:07 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2018-11-27 12:07:07 [INFO    ] Setting Beam 3 xbars
2018-11-27 12:07:07 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2018-11-27 12:07:08 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2018-11-27 12:07:08 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2018-11-27 12:07:08 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2018-11-27 12:07:08 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2018-11-27 12:07:08 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2018-11-27 12:07:08 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2018-11-27 12:07:08 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2018-11-27 12:07:08 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2018-11-27 12:07:08 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2018-11-27 12:07:08 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2018-11-27 12:07:09 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2018-11-27 12:07:09 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2018-11-27 12:07:09 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2018-11-27 12:07:09 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2018-11-27 12:07:09 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2018-11-27 12:07:09 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2018-11-27 12:07:09 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2018-11-27 12:07:09 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2018-11-27 12:07:09 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2018-11-27 12:07:10 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2018-11-27 12:07:10 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2018-11-27 12:07:10 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2018-11-27 12:07:10 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2018-11-27 12:07:10 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2018-11-27 12:07:10 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2018-11-27 12:07:10 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2018-11-27 12:07:10 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2018-11-27 12:07:10 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2018-11-27 12:07:11 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2018-11-27 12:07:11 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2018-11-27 12:07:11 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2018-11-27 12:07:11 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2018-11-27 12:07:11 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2018-11-27 12:07:11 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2018-11-27 12:07:11 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2018-11-27 12:07:11 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2018-11-27 12:07:11 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2018-11-27 12:07:12 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2018-11-27 12:07:12 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2018-11-27 12:07:12 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2018-11-27 12:07:12 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2018-11-27 12:07:12 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2018-11-27 12:07:12 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2018-11-27 12:07:12 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2018-11-27 12:07:12 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2018-11-27 12:07:12 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2018-11-27 12:07:12 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2018-11-27 12:07:13 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2018-11-27 12:07:13 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2018-11-27 12:07:13 [INFO    ] Setting Beam 4 xbars
2018-11-27 12:07:13 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2018-11-27 12:07:13 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 12:07:13 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 12:07:13 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2018-11-27 12:07:13 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2018-11-27 12:07:13 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2018-11-27 12:07:13 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2018-11-27 12:07:14 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2018-11-27 12:07:14 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2018-11-27 12:07:14 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2018-11-27 12:07:14 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2018-11-27 12:07:14 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2018-11-27 12:07:14 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2018-11-27 12:07:14 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2018-11-27 12:07:14 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2018-11-27 12:07:14 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2018-11-27 12:07:15 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2018-11-27 12:07:15 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2018-11-27 12:07:15 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2018-11-27 12:07:15 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2018-11-27 12:07:15 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2018-11-27 12:07:15 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2018-11-27 12:07:15 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2018-11-27 12:07:15 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2018-11-27 12:07:16 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2018-11-27 12:07:16 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2018-11-27 12:07:16 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2018-11-27 12:07:16 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2018-11-27 12:07:16 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2018-11-27 12:07:16 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2018-11-27 12:07:16 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2018-11-27 12:07:16 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2018-11-27 12:07:16 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2018-11-27 12:07:17 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2018-11-27 12:07:17 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2018-11-27 12:07:17 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2018-11-27 12:07:17 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2018-11-27 12:07:17 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2018-11-27 12:07:17 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2018-11-27 12:07:17 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2018-11-27 12:07:17 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2018-11-27 12:07:17 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2018-11-27 12:07:17 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2018-11-27 12:07:18 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2018-11-27 12:07:18 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2018-11-27 12:07:18 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2018-11-27 12:07:18 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2018-11-27 12:07:18 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2018-11-27 12:07:18 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2018-11-27 12:07:18 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2018-11-27 12:07:18 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2018-11-27 12:07:18 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2018-11-27 12:07:19 [INFO    ] All xbars set
2018-11-27 12:07:19 [INFO    ] Setting adder chain routing for all beams
2018-11-27 12:07:23 [INFO    ] Setting beam ids
2018-11-27 12:07:33 [INFO    ] Starting beam delay calibration
2018-11-27 12:07:45 [INFO    ] Beam 1 latency is 5719
2018-11-27 12:07:45 [INFO    ] Beam 2 latency is 5741
2018-11-27 12:07:45 [INFO    ] Beam 3 latency is 5733
2018-11-27 12:07:45 [INFO    ] Beam 4 latency is 5728
2018-11-27 12:07:45 [INFO    ] Stop calibration
2018-11-27 12:07:53 [INFO    ] Beam delay calibration successful
2018-11-27 12:07:53 [INFO    ] Calibration passed
2018-11-27 12:07:53 [DEBUG   ] spawn_process: at 2018-11-27 12:07:53 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2018-11-27 12:07:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2018-11-27 12:07:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2018-11-27 12:07:53 [DEBUG   ] spawn_process: at 2018-11-27 12:07:53 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2018-11-27 12:07:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2018-11-27 12:07:53 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2018-11-27 12:07:54 [DEBUG   ] spawn_process: at 2018-11-27 12:07:54 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2018-11-27 12:07:54 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2018-11-27 12:07:54 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2018-11-27 12:07:54 [DEBUG   ] spawn_process: at 2018-11-27 12:07:54 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2018-11-27 12:07:54 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2018-11-27 12:07:54 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2018-11-27 12:07:54 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 12:07:54 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 278 ms (1543320474.280 s) with PID 30223 and time 1543320475
2018-11-27 12:07:54 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -2 msec
2018-11-27 12:07:54 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 369 ms (1543320474.371 s) with PID 30226 and time 1543320475
2018-11-27 12:07:55 [DEBUG   ] Finished updating ant stats in 0.876 seconds
2018-11-27 12:07:55 [INFO    ] Started antennas statistics background thread
2018-11-27 12:07:55 [INFO    ] Started board flags background thread
2018-11-27 12:07:55 [INFO    ] Started BAM background thread
2018-11-27 12:07:55 [INFO    ] Finished the INI process (ref. #643886) in 104.556 s
2018-11-27 12:07:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:07:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:07:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:07:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:07:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:07:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:07:56 [DEBUG   ] Finished updating board flags in 1.064 seconds
2018-11-27 12:08:04 [DEBUG   ] Got command RPT from MCS: ref #643891
2018-11-27 12:08:04 [DEBUG   ] summary = ERROR
2018-11-27 12:08:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643891  13 58449 43684801 A  ERRORERROR'
2018-11-27 12:08:04 [DEBUG   ] Got command RPT from MCS: ref #643892
2018-11-27 12:08:04 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:08:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643892  77 58449 43684993 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:08:05 [DEBUG   ] Got command RPT from MCS: ref #643893
2018-11-27 12:08:05 [DEBUG   ] summary = ERROR
2018-11-27 12:08:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643893  13 58449 43685197 A  ERRORERROR'
2018-11-27 12:08:05 [DEBUG   ] Got command RPT from MCS: ref #643894
2018-11-27 12:08:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:08:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643894  77 58449 43685401 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:08:05 [DEBUG   ] Got command RPT from MCS: ref #643895
2018-11-27 12:08:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:08:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643895  12 58449 43685605 A  ERRORB—™š'
2018-11-27 12:08:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:08:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:08:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:08:55 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:08:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:08:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:08:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:08:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:08:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:08:56 [DEBUG   ] Finished updating board flags in 0.989 seconds
2018-11-27 12:09:10 [DEBUG   ] Got command RPT from MCS: ref #643896
2018-11-27 12:09:10 [DEBUG   ] summary = ERROR
2018-11-27 12:09:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643896  13 58449 43750941 A  ERRORERROR'
2018-11-27 12:09:10 [DEBUG   ] Got command RPT from MCS: ref #643897
2018-11-27 12:09:10 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:09:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643897  77 58449 43750942 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:09:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:09:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:09:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:09:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2018-11-27 12:09:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:09:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:09:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:09:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:09:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:09:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:09:57 [DEBUG   ] Finished updating board flags in 1.844 seconds
2018-11-27 12:10:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:10:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:10:49 [DEBUG   ] Got command RPT from MCS: ref #643902
2018-11-27 12:10:49 [DEBUG   ] summary = ERROR
2018-11-27 12:10:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643902  13 58449 43849029 A  ERRORERROR'
2018-11-27 12:10:49 [DEBUG   ] Got command RPT from MCS: ref #643903
2018-11-27 12:10:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:10:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643903  77 58449 43849030 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:10:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:10:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:10:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 12:10:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:10:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:10:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:10:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:10:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:10:57 [DEBUG   ] Finished updating board flags in 1.836 seconds
2018-11-27 12:11:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:11:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:11:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:11:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:11:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:11:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:11:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:11:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:11:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:11:56 [DEBUG   ] Finished updating board flags in 1.186 seconds
2018-11-27 12:12:15 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2018-11-27 12:12:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:12:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:12:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 12:12:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:12:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:12:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:12:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:12:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:12:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:12:56 [DEBUG   ] Finished updating board flags in 1.032 seconds
2018-11-27 12:13:04 [DEBUG   ] Got command RPT from MCS: ref #643921
2018-11-27 12:13:04 [DEBUG   ] summary = ERROR
2018-11-27 12:13:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643921  13 58449 43984881 A  ERRORERROR'
2018-11-27 12:13:05 [DEBUG   ] Got command RPT from MCS: ref #643922
2018-11-27 12:13:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:13:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643922  77 58449 43985085 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:13:05 [DEBUG   ] Got command RPT from MCS: ref #643923
2018-11-27 12:13:05 [DEBUG   ] summary = ERROR
2018-11-27 12:13:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643923  13 58449 43985289 A  ERRORERROR'
2018-11-27 12:13:05 [DEBUG   ] Got command RPT from MCS: ref #643925
2018-11-27 12:13:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:13:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643925  77 58449 43985481 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:13:05 [DEBUG   ] Got command RPT from MCS: ref #643926
2018-11-27 12:13:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:13:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643926  12 58449 43985685 A  ERRORB˜™š'
2018-11-27 12:13:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:13:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:13:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 12:13:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:13:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:13:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:13:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:13:56 [WARNING ] (13, 1, 'bot')
2018-11-27 12:13:56 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:13:56 [DEBUG   ] Finished updating board flags in 0.949 seconds
2018-11-27 12:14:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:14:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 12:14:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 12:14:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:14:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:14:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:14:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:14:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:14:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:14:57 [DEBUG   ] Finished updating board flags in 1.375 seconds
2018-11-27 12:15:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:15:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 12:15:49 [DEBUG   ] Got command RPT from MCS: ref #643935
2018-11-27 12:15:49 [DEBUG   ] summary = ERROR
2018-11-27 12:15:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643935  13 58449 44149022 A  ERRORERROR'
2018-11-27 12:15:49 [DEBUG   ] Got command RPT from MCS: ref #643936
2018-11-27 12:15:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:15:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643936  77 58449 44149022 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:15:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:15:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:15:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:15:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:15:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:15:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:15:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:15:57 [DEBUG   ] Finished updating board flags in 1.181 seconds
2018-11-27 12:16:15 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 12:16:35 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:16:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:16:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:16:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:16:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:16:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:16:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:16:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:16:57 [DEBUG   ] Finished updating board flags in 1.016 seconds
2018-11-27 12:17:15 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:17:35 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:17:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:17:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:17:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:17:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:17:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:17:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:17:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:17:57 [DEBUG   ] Finished updating board flags in 1.026 seconds
2018-11-27 12:18:04 [DEBUG   ] Got command RPT from MCS: ref #643956
2018-11-27 12:18:04 [DEBUG   ] summary = ERROR
2018-11-27 12:18:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643956  13 58449 44284973 A  ERRORERROR'
2018-11-27 12:18:05 [DEBUG   ] Got command RPT from MCS: ref #643957
2018-11-27 12:18:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:18:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643957  77 58449 44285177 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:18:05 [DEBUG   ] Got command RPT from MCS: ref #643958
2018-11-27 12:18:05 [DEBUG   ] summary = ERROR
2018-11-27 12:18:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643958  13 58449 44285369 A  ERRORERROR'
2018-11-27 12:18:05 [DEBUG   ] Got command RPT from MCS: ref #643959
2018-11-27 12:18:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:18:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643959  77 58449 44285573 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:18:05 [DEBUG   ] Got command RPT from MCS: ref #643960
2018-11-27 12:18:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:18:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643960  12 58449 44285777 A  ERRORB˜™š'
2018-11-27 12:18:15 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:18:36 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2018-11-27 12:18:55 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:18:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:18:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:18:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:18:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:18:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:18:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:18:58 [DEBUG   ] Finished updating board flags in 2.034 seconds
2018-11-27 12:19:16 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2018-11-27 12:19:36 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-27 12:19:56 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 12:19:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 12:19:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:19:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:19:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:19:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:19:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:19:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:19:57 [DEBUG   ] Finished updating board flags in 1.368 seconds
2018-11-27 12:20:16 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2018-11-27 12:20:36 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:20:49 [DEBUG   ] Got command RPT from MCS: ref #643965
2018-11-27 12:20:49 [DEBUG   ] summary = ERROR
2018-11-27 12:20:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643965  13 58449 44449029 A  ERRORERROR'
2018-11-27 12:20:49 [DEBUG   ] Got command RPT from MCS: ref #643966
2018-11-27 12:20:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:20:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643966  77 58449 44449030 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:20:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:20:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:20:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:20:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:20:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:20:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:20:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:20:57 [DEBUG   ] Finished updating board flags in 1.241 seconds
2018-11-27 12:21:16 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2018-11-27 12:21:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:21:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:21:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:21:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:21:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:21:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:21:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:21:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:21:57 [DEBUG   ] Finished updating board flags in 1.162 seconds
2018-11-27 12:22:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:22:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:22:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:22:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:22:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:22:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:22:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:22:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:22:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:22:57 [DEBUG   ] Finished updating board flags in 1.041 seconds
2018-11-27 12:23:05 [DEBUG   ] Got command RPT from MCS: ref #643988
2018-11-27 12:23:05 [DEBUG   ] summary = ERROR
2018-11-27 12:23:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643988  13 58449 44585061 A  ERRORERROR'
2018-11-27 12:23:05 [DEBUG   ] Got command RPT from MCS: ref #643989
2018-11-27 12:23:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:23:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643989  77 58449 44585265 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:23:05 [DEBUG   ] Got command RPT from MCS: ref #643990
2018-11-27 12:23:05 [DEBUG   ] summary = ERROR
2018-11-27 12:23:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643990  13 58449 44585457 A  ERRORERROR'
2018-11-27 12:23:05 [DEBUG   ] Got command RPT from MCS: ref #643992
2018-11-27 12:23:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:23:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643992  77 58449 44585661 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:23:05 [DEBUG   ] Got command RPT from MCS: ref #643993
2018-11-27 12:23:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:23:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   643993  12 58449 44585865 A  ERRORB™™š'
2018-11-27 12:23:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:23:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:23:52 [DEBUG   ] Got command DRX from MCS: ref #644002
2018-11-27 12:23:52 [INFO    ] sending drx command to lwa16
2018-11-27 12:23:52 [DEBUG   ] run_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 64500000.0 7 9 0']
2018-11-27 12:23:52 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 12:23:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   644002   8 58449 44632009 A  ERROR'
2018-11-27 12:23:52 [DEBUG   ] Got command DRX from MCS: ref #644003
2018-11-27 12:23:52 [INFO    ] sending drx command to lwa34
2018-11-27 12:23:52 [DEBUG   ] run_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 1 35100000.0 7 9 0']
2018-11-27 12:23:52 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 12:23:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   644003   8 58449 44632013 A  ERROR'
2018-11-27 12:23:52 [DEBUG   ] Got command DRX from MCS: ref #644004
2018-11-27 12:23:52 [INFO    ] sending drx command to lwa16
2018-11-27 12:23:52 [DEBUG   ] run_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 79200000.0 7 9 0']
2018-11-27 12:23:52 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 12:23:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   644004   8 58449 44632017 A  ERROR'
2018-11-27 12:23:52 [DEBUG   ] Got command DRX from MCS: ref #644005
2018-11-27 12:23:52 [INFO    ] sending drx command to lwa34
2018-11-27 12:23:52 [DEBUG   ] run_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 2 49800000.0 7 9 0']
2018-11-27 12:23:52 [DEBUG   ] drx = exited with status True, code 0x00
2018-11-27 12:23:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   644005   8 58449 44632021 A  ERROR'
2018-11-27 12:23:52 [DEBUG   ] Got command BAM from MCS: ref #644006
2018-11-27 12:23:52 [INFO    ] Finished processing BAM request
2018-11-27 12:23:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:23:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644006   8 58449 44632025 A  ERROR'
2018-11-27 12:23:52 [DEBUG   ] Got command BAM from MCS: ref #644007
2018-11-27 12:23:52 [INFO    ] Finished processing BAM request
2018-11-27 12:23:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:23:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644007   8 58449 44632029 A  ERROR'
2018-11-27 12:23:52 [INFO    ] Finished starting DRX
2018-11-27 12:23:52 [INFO    ] Finished starting DRX
2018-11-27 12:23:52 [INFO    ] Finished starting DRX
2018-11-27 12:23:52 [INFO    ] Finished starting DRX
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:23:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:23:52 [DEBUG   ] spawn_process: at 2018-11-27 12:23:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:23:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:23:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:23:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:23:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:23:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:23:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:23:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:23:57 [DEBUG   ] Finished updating board flags in 1.013 seconds
2018-11-27 12:24:14 [DEBUG   ] Got command BAM from MCS: ref #644011
2018-11-27 12:24:14 [INFO    ] Finished processing BAM request
2018-11-27 12:24:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:24:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644011   8 58449 44654033 A  ERROR'
2018-11-27 12:24:14 [DEBUG   ] Got command BAM from MCS: ref #644010
2018-11-27 12:24:14 [INFO    ] Finished processing BAM request
2018-11-27 12:24:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:24:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644010   8 58449 44654037 A  ERROR'
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:24:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:24:14 [DEBUG   ] spawn_process: at 2018-11-27 12:24:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:24:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:24:36 [DEBUG   ] Got command BAM from MCS: ref #644013
2018-11-27 12:24:36 [INFO    ] Finished processing BAM request
2018-11-27 12:24:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:24:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644013   8 58449 44676025 A  ERROR'
2018-11-27 12:24:36 [DEBUG   ] Got command BAM from MCS: ref #644012
2018-11-27 12:24:36 [INFO    ] Finished processing BAM request
2018-11-27 12:24:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:24:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644012   8 58449 44676029 A  ERROR'
2018-11-27 12:24:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:24:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:24:36 [DEBUG   ] spawn_process: at 2018-11-27 12:24:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:24:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:24:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:24:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:24:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:24:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:24:57 [WARNING ] (13, 1, 'bot')
2018-11-27 12:24:57 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:24:57 [DEBUG   ] Finished updating board flags in 0.970 seconds
2018-11-27 12:24:58 [DEBUG   ] Got command BAM from MCS: ref #644015
2018-11-27 12:24:58 [INFO    ] Finished processing BAM request
2018-11-27 12:24:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:24:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644015   8 58449 44698029 A  ERROR'
2018-11-27 12:24:58 [DEBUG   ] Got command BAM from MCS: ref #644014
2018-11-27 12:24:58 [INFO    ] Finished processing BAM request
2018-11-27 12:24:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:24:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644014   8 58449 44698033 A  ERROR'
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:24:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:24:58 [DEBUG   ] spawn_process: at 2018-11-27 12:24:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:25:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:25:20 [DEBUG   ] Got command BAM from MCS: ref #644021
2018-11-27 12:25:20 [INFO    ] Finished processing BAM request
2018-11-27 12:25:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:25:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644021   8 58449 44720025 A  ERROR'
2018-11-27 12:25:20 [DEBUG   ] Got command BAM from MCS: ref #644020
2018-11-27 12:25:20 [INFO    ] Finished processing BAM request
2018-11-27 12:25:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:25:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644020   8 58449 44720029 A  ERROR'
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:25:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:25:20 [DEBUG   ] spawn_process: at 2018-11-27 12:25:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:25:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:25:42 [DEBUG   ] Got command BAM from MCS: ref #644023
2018-11-27 12:25:42 [INFO    ] Finished processing BAM request
2018-11-27 12:25:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:25:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644023   8 58449 44742025 A  ERROR'
2018-11-27 12:25:42 [DEBUG   ] Got command BAM from MCS: ref #644022
2018-11-27 12:25:42 [INFO    ] Finished processing BAM request
2018-11-27 12:25:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:25:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644022   8 58449 44742029 A  ERROR'
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:25:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:25:42 [DEBUG   ] spawn_process: at 2018-11-27 12:25:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:25:49 [DEBUG   ] Got command RPT from MCS: ref #644024
2018-11-27 12:25:49 [DEBUG   ] summary = ERROR
2018-11-27 12:25:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644024  13 58449 44749029 A  ERRORERROR'
2018-11-27 12:25:49 [DEBUG   ] Got command RPT from MCS: ref #644025
2018-11-27 12:25:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:25:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644025  77 58449 44749030 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:25:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:25:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:25:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 12:25:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:25:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa7, chip 3
2018-11-27 12:25:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:25:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:25:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:25:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:25:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:25:58 [DEBUG   ] Finished updating board flags in 1.355 seconds
2018-11-27 12:26:04 [DEBUG   ] Got command BAM from MCS: ref #644042
2018-11-27 12:26:04 [INFO    ] Finished processing BAM request
2018-11-27 12:26:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:26:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644042   8 58449 44764029 A  ERROR'
2018-11-27 12:26:04 [DEBUG   ] Got command BAM from MCS: ref #644041
2018-11-27 12:26:04 [INFO    ] Finished processing BAM request
2018-11-27 12:26:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:26:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644041   8 58449 44764033 A  ERROR'
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:26:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:26:04 [DEBUG   ] spawn_process: at 2018-11-27 12:26:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:26:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:26:26 [DEBUG   ] Got command BAM from MCS: ref #644043
2018-11-27 12:26:26 [INFO    ] Finished processing BAM request
2018-11-27 12:26:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:26:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644043   8 58449 44786037 A  ERROR'
2018-11-27 12:26:26 [DEBUG   ] Got command BAM from MCS: ref #644044
2018-11-27 12:26:26 [INFO    ] Finished processing BAM request
2018-11-27 12:26:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:26:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644044   8 58449 44786041 A  ERROR'
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:26:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:26:26 [DEBUG   ] spawn_process: at 2018-11-27 12:26:26 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:26:36 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 12:26:48 [DEBUG   ] Got command BAM from MCS: ref #644045
2018-11-27 12:26:48 [INFO    ] Finished processing BAM request
2018-11-27 12:26:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:26:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644045   8 58449 44808025 A  ERROR'
2018-11-27 12:26:48 [DEBUG   ] Got command BAM from MCS: ref #644046
2018-11-27 12:26:48 [INFO    ] Finished processing BAM request
2018-11-27 12:26:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:26:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644046   8 58449 44808029 A  ERROR'
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:26:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:26:48 [DEBUG   ] spawn_process: at 2018-11-27 12:26:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:26:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:26:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:26:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 12:26:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:26:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:26:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:26:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:26:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:26:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:26:58 [DEBUG   ] Finished updating board flags in 1.212 seconds
2018-11-27 12:27:10 [DEBUG   ] Got command BAM from MCS: ref #644047
2018-11-27 12:27:10 [INFO    ] Finished processing BAM request
2018-11-27 12:27:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:27:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644047   8 58449 44830029 A  ERROR'
2018-11-27 12:27:10 [DEBUG   ] Got command BAM from MCS: ref #644048
2018-11-27 12:27:10 [INFO    ] Finished processing BAM request
2018-11-27 12:27:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:27:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644048   8 58449 44830033 A  ERROR'
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:27:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:27:10 [DEBUG   ] spawn_process: at 2018-11-27 12:27:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:27:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:27:32 [DEBUG   ] Got command BAM from MCS: ref #644050
2018-11-27 12:27:32 [INFO    ] Finished processing BAM request
2018-11-27 12:27:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:27:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644050   8 58449 44852025 A  ERROR'
2018-11-27 12:27:32 [DEBUG   ] Got command BAM from MCS: ref #644049
2018-11-27 12:27:32 [INFO    ] Finished processing BAM request
2018-11-27 12:27:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:27:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644049   8 58449 44852029 A  ERROR'
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:27:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:27:32 [DEBUG   ] spawn_process: at 2018-11-27 12:27:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:27:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:27:54 [DEBUG   ] Got command BAM from MCS: ref #644052
2018-11-27 12:27:54 [INFO    ] Finished processing BAM request
2018-11-27 12:27:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:27:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644052   8 58449 44874033 A  ERROR'
2018-11-27 12:27:54 [DEBUG   ] Got command BAM from MCS: ref #644051
2018-11-27 12:27:54 [INFO    ] Finished processing BAM request
2018-11-27 12:27:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:27:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644051   8 58449 44874037 A  ERROR'
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:27:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:27:54 [DEBUG   ] spawn_process: at 2018-11-27 12:27:54 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:27:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:27:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:27:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 12:27:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:27:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:27:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:27:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:27:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:27:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:27:58 [DEBUG   ] Finished updating board flags in 1.201 seconds
2018-11-27 12:28:05 [DEBUG   ] Got command RPT from MCS: ref #644053
2018-11-27 12:28:05 [DEBUG   ] summary = ERROR
2018-11-27 12:28:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644053  13 58449 44885141 A  ERRORERROR'
2018-11-27 12:28:05 [DEBUG   ] Got command RPT from MCS: ref #644054
2018-11-27 12:28:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:28:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644054  77 58449 44885345 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:28:05 [DEBUG   ] Got command RPT from MCS: ref #644055
2018-11-27 12:28:05 [DEBUG   ] summary = ERROR
2018-11-27 12:28:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644055  13 58449 44885549 A  ERRORERROR'
2018-11-27 12:28:05 [DEBUG   ] Got command RPT from MCS: ref #644056
2018-11-27 12:28:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:28:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644056  77 58449 44885753 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:28:05 [DEBUG   ] Got command RPT from MCS: ref #644057
2018-11-27 12:28:05 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:28:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644057  12 58449 44885945 A  ERRORB ff'
2018-11-27 12:28:16 [DEBUG   ] Got command BAM from MCS: ref #644058
2018-11-27 12:28:16 [INFO    ] Finished processing BAM request
2018-11-27 12:28:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:28:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644058   8 58449 44896033 A  ERROR'
2018-11-27 12:28:16 [DEBUG   ] Got command BAM from MCS: ref #644059
2018-11-27 12:28:16 [INFO    ] Finished processing BAM request
2018-11-27 12:28:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:28:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644059   8 58449 44896037 A  ERROR'
2018-11-27 12:28:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:28:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:28:16 [DEBUG   ] spawn_process: at 2018-11-27 12:28:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:28:36 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:28:38 [DEBUG   ] Got command BAM from MCS: ref #644064
2018-11-27 12:28:38 [INFO    ] Finished processing BAM request
2018-11-27 12:28:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:28:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644064   8 58449 44918029 A  ERROR'
2018-11-27 12:28:38 [DEBUG   ] Got command BAM from MCS: ref #644065
2018-11-27 12:28:38 [INFO    ] Finished processing BAM request
2018-11-27 12:28:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:28:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644065   8 58449 44918033 A  ERROR'
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:28:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:28:38 [DEBUG   ] spawn_process: at 2018-11-27 12:28:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:28:56 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2018-11-27 12:28:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:28:58 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:28:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:28:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:28:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:28:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:28:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:28:58 [DEBUG   ] Finished updating board flags in 1.194 seconds
2018-11-27 12:29:00 [DEBUG   ] Got command BAM from MCS: ref #644067
2018-11-27 12:29:00 [INFO    ] Finished processing BAM request
2018-11-27 12:29:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:29:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644067   8 58449 44940029 A  ERROR'
2018-11-27 12:29:00 [DEBUG   ] Got command BAM from MCS: ref #644066
2018-11-27 12:29:00 [INFO    ] Finished processing BAM request
2018-11-27 12:29:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:29:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644066   8 58449 44940033 A  ERROR'
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:29:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:29:00 [DEBUG   ] spawn_process: at 2018-11-27 12:29:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:29:16 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2018-11-27 12:29:22 [DEBUG   ] Got command BAM from MCS: ref #644069
2018-11-27 12:29:22 [INFO    ] Finished processing BAM request
2018-11-27 12:29:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:29:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644069   8 58449 44962033 A  ERROR'
2018-11-27 12:29:22 [DEBUG   ] Got command BAM from MCS: ref #644068
2018-11-27 12:29:22 [INFO    ] Finished processing BAM request
2018-11-27 12:29:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:29:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644068   8 58449 44962037 A  ERROR'
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:29:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:29:22 [DEBUG   ] spawn_process: at 2018-11-27 12:29:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:29:36 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:29:44 [DEBUG   ] Got command BAM from MCS: ref #644070
2018-11-27 12:29:44 [INFO    ] Finished processing BAM request
2018-11-27 12:29:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:29:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644070   8 58449 44984025 A  ERROR'
2018-11-27 12:29:44 [DEBUG   ] Got command BAM from MCS: ref #644071
2018-11-27 12:29:44 [INFO    ] Finished processing BAM request
2018-11-27 12:29:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:29:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644071   8 58449 44984029 A  ERROR'
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:29:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:29:44 [DEBUG   ] spawn_process: at 2018-11-27 12:29:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:29:56 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:29:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 12:29:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:29:58 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:29:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:29:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:29:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:29:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:29:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:29:58 [DEBUG   ] Finished updating board flags in 1.006 seconds
2018-11-27 12:30:06 [DEBUG   ] Got command BAM from MCS: ref #644072
2018-11-27 12:30:06 [INFO    ] Finished processing BAM request
2018-11-27 12:30:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:30:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644072   8 58449 45006029 A  ERROR'
2018-11-27 12:30:06 [DEBUG   ] Got command BAM from MCS: ref #644073
2018-11-27 12:30:06 [INFO    ] Finished processing BAM request
2018-11-27 12:30:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:30:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644073   8 58449 45006033 A  ERROR'
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:30:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:30:06 [DEBUG   ] spawn_process: at 2018-11-27 12:30:06 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:30:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:30:28 [DEBUG   ] Got command BAM from MCS: ref #644074
2018-11-27 12:30:28 [INFO    ] Finished processing BAM request
2018-11-27 12:30:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:30:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644074   8 58449 45028033 A  ERROR'
2018-11-27 12:30:28 [DEBUG   ] Got command BAM from MCS: ref #644075
2018-11-27 12:30:28 [INFO    ] Finished processing BAM request
2018-11-27 12:30:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:30:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644075   8 58449 45028037 A  ERROR'
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:30:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:30:28 [DEBUG   ] spawn_process: at 2018-11-27 12:30:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:30:36 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:30:49 [DEBUG   ] Got command RPT from MCS: ref #644078
2018-11-27 12:30:49 [DEBUG   ] summary = ERROR
2018-11-27 12:30:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644078  13 58449 45049033 A  ERRORERROR'
2018-11-27 12:30:49 [DEBUG   ] Got command RPT from MCS: ref #644079
2018-11-27 12:30:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:30:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644079  77 58449 45049034 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:30:50 [DEBUG   ] Got command BAM from MCS: ref #644076
2018-11-27 12:30:50 [INFO    ] Finished processing BAM request
2018-11-27 12:30:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:30:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644076   8 58449 45050033 A  ERROR'
2018-11-27 12:30:50 [DEBUG   ] Got command BAM from MCS: ref #644077
2018-11-27 12:30:50 [INFO    ] Finished processing BAM request
2018-11-27 12:30:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:30:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644077   8 58449 45050037 A  ERROR'
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:30:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:30:50 [DEBUG   ] spawn_process: at 2018-11-27 12:30:50 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:30:56 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:30:57 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:30:58 [WARNING ] MonitorFlags: Temperature 80.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:30:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:30:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:30:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:30:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:30:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:30:58 [DEBUG   ] Finished updating board flags in 1.055 seconds
2018-11-27 12:31:12 [DEBUG   ] Got command BAM from MCS: ref #644096
2018-11-27 12:31:12 [INFO    ] Finished processing BAM request
2018-11-27 12:31:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:31:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644096   8 58449 45072029 A  ERROR'
2018-11-27 12:31:12 [DEBUG   ] Got command BAM from MCS: ref #644095
2018-11-27 12:31:12 [INFO    ] Finished processing BAM request
2018-11-27 12:31:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:31:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644095   8 58449 45072033 A  ERROR'
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:31:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:31:12 [DEBUG   ] spawn_process: at 2018-11-27 12:31:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:31:16 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:31:34 [DEBUG   ] Got command BAM from MCS: ref #644102
2018-11-27 12:31:34 [INFO    ] Finished processing BAM request
2018-11-27 12:31:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:31:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644102   8 58449 45094033 A  ERROR'
2018-11-27 12:31:34 [DEBUG   ] Got command BAM from MCS: ref #644101
2018-11-27 12:31:34 [INFO    ] Finished processing BAM request
2018-11-27 12:31:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:31:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644101   8 58449 45094037 A  ERROR'
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:31:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:31:34 [DEBUG   ] spawn_process: at 2018-11-27 12:31:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:31:36 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:31:56 [DEBUG   ] Got command BAM from MCS: ref #644103
2018-11-27 12:31:56 [INFO    ] Finished processing BAM request
2018-11-27 12:31:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:31:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644103   8 58449 45116029 A  ERROR'
2018-11-27 12:31:56 [DEBUG   ] Got command BAM from MCS: ref #644104
2018-11-27 12:31:56 [INFO    ] Finished processing BAM request
2018-11-27 12:31:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:31:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644104   8 58449 45116033 A  ERROR'
2018-11-27 12:31:56 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:31:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:31:56 [DEBUG   ] spawn_process: at 2018-11-27 12:31:56 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:31:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2018-11-27 12:31:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:31:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2018-11-27 12:31:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:31:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:31:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:31:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:31:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:31:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:31:59 [DEBUG   ] Finished updating board flags in 2.012 seconds
2018-11-27 12:32:16 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:32:18 [DEBUG   ] Got command BAM from MCS: ref #644106
2018-11-27 12:32:18 [INFO    ] Finished processing BAM request
2018-11-27 12:32:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:32:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644106   8 58449 45138029 A  ERROR'
2018-11-27 12:32:18 [DEBUG   ] Got command BAM from MCS: ref #644105
2018-11-27 12:32:18 [INFO    ] Finished processing BAM request
2018-11-27 12:32:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:32:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644105   8 58449 45138033 A  ERROR'
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:32:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:32:18 [DEBUG   ] spawn_process: at 2018-11-27 12:32:18 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:32:36 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:32:40 [DEBUG   ] Got command BAM from MCS: ref #644108
2018-11-27 12:32:40 [INFO    ] Finished processing BAM request
2018-11-27 12:32:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:32:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644108   8 58449 45160033 A  ERROR'
2018-11-27 12:32:40 [DEBUG   ] Got command BAM from MCS: ref #644107
2018-11-27 12:32:40 [INFO    ] Finished processing BAM request
2018-11-27 12:32:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:32:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644107   8 58449 45160037 A  ERROR'
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:32:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:32:40 [DEBUG   ] spawn_process: at 2018-11-27 12:32:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:32:56 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:32:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 12:32:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 12:32:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 12:32:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:32:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:32:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:32:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:32:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:32:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:32:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:32:58 [DEBUG   ] Finished updating board flags in 1.377 seconds
2018-11-27 12:33:02 [DEBUG   ] Got command BAM from MCS: ref #644111
2018-11-27 12:33:02 [INFO    ] Finished processing BAM request
2018-11-27 12:33:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:33:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644111   8 58449 45182033 A  ERROR'
2018-11-27 12:33:02 [DEBUG   ] Got command BAM from MCS: ref #644112
2018-11-27 12:33:02 [INFO    ] Finished processing BAM request
2018-11-27 12:33:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:33:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644112   8 58449 45182037 A  ERROR'
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:33:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:33:02 [DEBUG   ] spawn_process: at 2018-11-27 12:33:02 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:33:05 [DEBUG   ] Got command RPT from MCS: ref #644113
2018-11-27 12:33:05 [DEBUG   ] summary = ERROR
2018-11-27 12:33:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644113  13 58449 45185229 A  ERRORERROR'
2018-11-27 12:33:05 [DEBUG   ] Got command RPT from MCS: ref #644114
2018-11-27 12:33:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:33:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644114  77 58449 45185433 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:33:05 [DEBUG   ] Got command RPT from MCS: ref #644115
2018-11-27 12:33:05 [DEBUG   ] summary = ERROR
2018-11-27 12:33:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644115  13 58449 45185625 A  ERRORERROR'
2018-11-27 12:33:05 [DEBUG   ] Got command RPT from MCS: ref #644117
2018-11-27 12:33:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:33:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644117  77 58449 45185829 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:33:06 [DEBUG   ] Got command RPT from MCS: ref #644119
2018-11-27 12:33:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:33:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644119  12 58449 45186033 A  ERRORB¢ff'
2018-11-27 12:33:16 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:33:24 [DEBUG   ] Got command BAM from MCS: ref #644124
2018-11-27 12:33:24 [INFO    ] Finished processing BAM request
2018-11-27 12:33:24 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:33:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644124   8 58449 45204029 A  ERROR'
2018-11-27 12:33:24 [DEBUG   ] Got command BAM from MCS: ref #644123
2018-11-27 12:33:24 [INFO    ] Finished processing BAM request
2018-11-27 12:33:24 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:33:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644123   8 58449 45204033 A  ERROR'
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:33:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:33:24 [DEBUG   ] spawn_process: at 2018-11-27 12:33:24 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:33:36 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:33:46 [DEBUG   ] Got command BAM from MCS: ref #644125
2018-11-27 12:33:46 [INFO    ] Finished processing BAM request
2018-11-27 12:33:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:33:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644125   8 58449 45226025 A  ERROR'
2018-11-27 12:33:46 [DEBUG   ] Got command BAM from MCS: ref #644126
2018-11-27 12:33:46 [INFO    ] Finished processing BAM request
2018-11-27 12:33:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:33:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644126   8 58449 45226029 A  ERROR'
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:33:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:33:46 [DEBUG   ] spawn_process: at 2018-11-27 12:33:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:33:56 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:33:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2018-11-27 12:33:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:33:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:33:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:33:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:33:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:33:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:33:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:33:58 [DEBUG   ] Finished updating board flags in 1.234 seconds
2018-11-27 12:34:08 [DEBUG   ] Got command BAM from MCS: ref #644128
2018-11-27 12:34:08 [INFO    ] Finished processing BAM request
2018-11-27 12:34:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:34:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644128   8 58449 45248029 A  ERROR'
2018-11-27 12:34:08 [DEBUG   ] Got command BAM from MCS: ref #644127
2018-11-27 12:34:08 [INFO    ] Finished processing BAM request
2018-11-27 12:34:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:34:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644127   8 58449 45248033 A  ERROR'
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:34:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:34:08 [DEBUG   ] spawn_process: at 2018-11-27 12:34:08 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:34:16 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:34:30 [DEBUG   ] Got command BAM from MCS: ref #644133
2018-11-27 12:34:30 [INFO    ] Finished processing BAM request
2018-11-27 12:34:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:34:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644133   8 58449 45270033 A  ERROR'
2018-11-27 12:34:30 [DEBUG   ] Got command BAM from MCS: ref #644134
2018-11-27 12:34:30 [INFO    ] Finished processing BAM request
2018-11-27 12:34:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:34:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644134   8 58449 45270037 A  ERROR'
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:34:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:34:30 [DEBUG   ] spawn_process: at 2018-11-27 12:34:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:34:36 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2018-11-27 12:34:52 [DEBUG   ] Got command BAM from MCS: ref #644135
2018-11-27 12:34:52 [INFO    ] Finished processing BAM request
2018-11-27 12:34:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:34:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644135   8 58449 45292033 A  ERROR'
2018-11-27 12:34:52 [DEBUG   ] Got command BAM from MCS: ref #644136
2018-11-27 12:34:52 [INFO    ] Finished processing BAM request
2018-11-27 12:34:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:34:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644136   8 58449 45292037 A  ERROR'
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:34:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:34:52 [DEBUG   ] spawn_process: at 2018-11-27 12:34:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:34:56 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 12:34:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:34:58 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:34:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:34:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:34:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:34:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:34:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:34:58 [DEBUG   ] Finished updating board flags in 1.171 seconds
2018-11-27 12:35:14 [DEBUG   ] Got command BAM from MCS: ref #644137
2018-11-27 12:35:14 [INFO    ] Finished processing BAM request
2018-11-27 12:35:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:35:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644137   8 58449 45314029 A  ERROR'
2018-11-27 12:35:14 [DEBUG   ] Got command BAM from MCS: ref #644138
2018-11-27 12:35:14 [INFO    ] Finished processing BAM request
2018-11-27 12:35:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:35:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644138   8 58449 45314033 A  ERROR'
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:35:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:35:14 [DEBUG   ] spawn_process: at 2018-11-27 12:35:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:35:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 12:35:36 [DEBUG   ] Got command BAM from MCS: ref #644140
2018-11-27 12:35:36 [INFO    ] Finished processing BAM request
2018-11-27 12:35:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:35:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644140   8 58449 45336025 A  ERROR'
2018-11-27 12:35:36 [DEBUG   ] Got command BAM from MCS: ref #644139
2018-11-27 12:35:36 [INFO    ] Finished processing BAM request
2018-11-27 12:35:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:35:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644139   8 58449 45336029 A  ERROR'
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:35:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:35:36 [DEBUG   ] spawn_process: at 2018-11-27 12:35:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:35:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 12:35:49 [DEBUG   ] Got command RPT from MCS: ref #644141
2018-11-27 12:35:49 [DEBUG   ] summary = ERROR
2018-11-27 12:35:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644141  13 58449 45349029 A  ERRORERROR'
2018-11-27 12:35:49 [DEBUG   ] Got command RPT from MCS: ref #644142
2018-11-27 12:35:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:35:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644142  77 58449 45349030 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:35:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 12:35:58 [DEBUG   ] Got command BAM from MCS: ref #644159
2018-11-27 12:35:58 [INFO    ] Finished processing BAM request
2018-11-27 12:35:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644159   8 58449 45358025 A  ERROR'
2018-11-27 12:35:58 [DEBUG   ] Got command BAM from MCS: ref #644158
2018-11-27 12:35:58 [INFO    ] Finished processing BAM request
2018-11-27 12:35:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:35:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644158   8 58449 45358029 A  ERROR'
2018-11-27 12:35:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:35:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 4
2018-11-27 12:35:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 12:35:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 12:35:58 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:35:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:35:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:35:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:35:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:35:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:35:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:35:58 [DEBUG   ] Finished updating board flags in 1.032 seconds
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:35:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:35:58 [DEBUG   ] spawn_process: at 2018-11-27 12:35:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:36:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 12:36:20 [DEBUG   ] Got command BAM from MCS: ref #644160
2018-11-27 12:36:20 [INFO    ] Finished processing BAM request
2018-11-27 12:36:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:36:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644160   8 58449 45380029 A  ERROR'
2018-11-27 12:36:20 [DEBUG   ] Got command BAM from MCS: ref #644161
2018-11-27 12:36:20 [INFO    ] Finished processing BAM request
2018-11-27 12:36:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:36:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644161   8 58449 45380033 A  ERROR'
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:36:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:36:20 [DEBUG   ] spawn_process: at 2018-11-27 12:36:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:36:37 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 12:36:42 [DEBUG   ] Got command BAM from MCS: ref #644163
2018-11-27 12:36:42 [INFO    ] Finished processing BAM request
2018-11-27 12:36:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:36:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644163   8 58449 45402033 A  ERROR'
2018-11-27 12:36:42 [DEBUG   ] Got command BAM from MCS: ref #644162
2018-11-27 12:36:42 [INFO    ] Finished processing BAM request
2018-11-27 12:36:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:36:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644162   8 58449 45402037 A  ERROR'
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:36:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:36:42 [DEBUG   ] spawn_process: at 2018-11-27 12:36:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:36:57 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 12:36:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 12:36:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 12:36:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:36:58 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:36:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:36:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:36:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:36:58 [WARNING ] (13, 1, 'bot')
2018-11-27 12:36:58 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:36:58 [DEBUG   ] Finished updating board flags in 0.990 seconds
2018-11-27 12:37:04 [DEBUG   ] Got command BAM from MCS: ref #644164
2018-11-27 12:37:04 [INFO    ] Finished processing BAM request
2018-11-27 12:37:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:37:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644164   8 58449 45424029 A  ERROR'
2018-11-27 12:37:04 [DEBUG   ] Got command BAM from MCS: ref #644165
2018-11-27 12:37:04 [INFO    ] Finished processing BAM request
2018-11-27 12:37:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:37:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644165   8 58449 45424033 A  ERROR'
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:37:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:37:04 [DEBUG   ] spawn_process: at 2018-11-27 12:37:04 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:37:17 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2018-11-27 12:37:26 [DEBUG   ] Got command BAM from MCS: ref #644170
2018-11-27 12:37:26 [INFO    ] Finished processing BAM request
2018-11-27 12:37:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:37:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644170   8 58449 45446033 A  ERROR'
2018-11-27 12:37:26 [DEBUG   ] Got command BAM from MCS: ref #644171
2018-11-27 12:37:26 [INFO    ] Finished processing BAM request
2018-11-27 12:37:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:37:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644171   8 58449 45446037 A  ERROR'
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:37:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:37:26 [DEBUG   ] spawn_process: at 2018-11-27 12:37:26 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:37:37 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2018-11-27 12:37:48 [DEBUG   ] Got command BAM from MCS: ref #644172
2018-11-27 12:37:48 [INFO    ] Finished processing BAM request
2018-11-27 12:37:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644172   8 58449 45468029 A  ERROR'
2018-11-27 12:37:48 [DEBUG   ] Got command BAM from MCS: ref #644173
2018-11-27 12:37:48 [INFO    ] Finished processing BAM request
2018-11-27 12:37:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:37:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644173   8 58449 45468033 A  ERROR'
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:37:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:37:48 [DEBUG   ] spawn_process: at 2018-11-27 12:37:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:37:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:37:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:37:59 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:37:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:37:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:37:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:37:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:37:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:37:59 [DEBUG   ] Finished updating board flags in 1.341 seconds
2018-11-27 12:38:05 [DEBUG   ] Got command RPT from MCS: ref #644174
2018-11-27 12:38:05 [DEBUG   ] summary = ERROR
2018-11-27 12:38:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644174  13 58449 45485321 A  ERRORERROR'
2018-11-27 12:38:05 [DEBUG   ] Got command RPT from MCS: ref #644175
2018-11-27 12:38:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:38:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644175  77 58449 45485513 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:38:05 [DEBUG   ] Got command RPT from MCS: ref #644176
2018-11-27 12:38:05 [DEBUG   ] summary = ERROR
2018-11-27 12:38:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644176  13 58449 45485717 A  ERRORERROR'
2018-11-27 12:38:05 [DEBUG   ] Got command RPT from MCS: ref #644177
2018-11-27 12:38:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:38:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644177  77 58449 45485921 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:38:06 [DEBUG   ] Got command RPT from MCS: ref #644178
2018-11-27 12:38:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:38:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644178  12 58449 45486125 A  ERRORB£ff'
2018-11-27 12:38:10 [DEBUG   ] Got command BAM from MCS: ref #644179
2018-11-27 12:38:10 [INFO    ] Finished processing BAM request
2018-11-27 12:38:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:38:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644179   8 58449 45490029 A  ERROR'
2018-11-27 12:38:10 [DEBUG   ] Got command BAM from MCS: ref #644180
2018-11-27 12:38:10 [INFO    ] Finished processing BAM request
2018-11-27 12:38:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:38:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644180   8 58449 45490033 A  ERROR'
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:38:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:38:10 [DEBUG   ] spawn_process: at 2018-11-27 12:38:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:38:17 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2018-11-27 12:38:32 [DEBUG   ] Got command BAM from MCS: ref #644181
2018-11-27 12:38:32 [INFO    ] Finished processing BAM request
2018-11-27 12:38:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:38:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644181   8 58449 45512029 A  ERROR'
2018-11-27 12:38:32 [DEBUG   ] Got command BAM from MCS: ref #644182
2018-11-27 12:38:32 [INFO    ] Finished processing BAM request
2018-11-27 12:38:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:38:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644182   8 58449 45512033 A  ERROR'
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:38:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:38:32 [DEBUG   ] spawn_process: at 2018-11-27 12:38:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:38:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:38:54 [DEBUG   ] Got command BAM from MCS: ref #644184
2018-11-27 12:38:54 [INFO    ] Finished processing BAM request
2018-11-27 12:38:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:38:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644184   8 58449 45534033 A  ERROR'
2018-11-27 12:38:54 [DEBUG   ] Got command BAM from MCS: ref #644183
2018-11-27 12:38:54 [INFO    ] Finished processing BAM request
2018-11-27 12:38:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:38:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644183   8 58449 45534037 A  ERROR'
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:38:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:38:54 [DEBUG   ] spawn_process: at 2018-11-27 12:38:54 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:38:57 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2018-11-27 12:38:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 12:38:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 12:38:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:38:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 12:38:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:38:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:38:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:38:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:38:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:38:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:38:59 [DEBUG   ] Finished updating board flags in 1.198 seconds
2018-11-27 12:39:16 [DEBUG   ] Got command BAM from MCS: ref #644185
2018-11-27 12:39:16 [INFO    ] Finished processing BAM request
2018-11-27 12:39:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:39:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644185   8 58449 45556033 A  ERROR'
2018-11-27 12:39:16 [DEBUG   ] Got command BAM from MCS: ref #644186
2018-11-27 12:39:16 [INFO    ] Finished processing BAM request
2018-11-27 12:39:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:39:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644186   8 58449 45556037 A  ERROR'
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:39:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:39:16 [DEBUG   ] spawn_process: at 2018-11-27 12:39:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:39:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:39:37 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2018-11-27 12:39:38 [DEBUG   ] Got command BAM from MCS: ref #644187
2018-11-27 12:39:38 [INFO    ] Finished processing BAM request
2018-11-27 12:39:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:39:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644187   8 58449 45578025 A  ERROR'
2018-11-27 12:39:38 [DEBUG   ] Got command BAM from MCS: ref #644188
2018-11-27 12:39:38 [INFO    ] Finished processing BAM request
2018-11-27 12:39:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:39:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644188   8 58449 45578029 A  ERROR'
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:39:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:39:38 [DEBUG   ] spawn_process: at 2018-11-27 12:39:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:39:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:39:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 12:39:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 12:39:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:39:59 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:39:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 12:39:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:39:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:39:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:39:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:39:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:39:59 [DEBUG   ] Finished updating board flags in 1.011 seconds
2018-11-27 12:40:00 [DEBUG   ] Got command BAM from MCS: ref #644189
2018-11-27 12:40:00 [INFO    ] Finished processing BAM request
2018-11-27 12:40:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:40:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644189   8 58449 45600034 A  ERROR'
2018-11-27 12:40:00 [DEBUG   ] Got command BAM from MCS: ref #644190
2018-11-27 12:40:00 [INFO    ] Finished processing BAM request
2018-11-27 12:40:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:40:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644190   8 58449 45600038 A  ERROR'
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:40:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:40:00 [DEBUG   ] spawn_process: at 2018-11-27 12:40:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:40:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:40:22 [DEBUG   ] Got command BAM from MCS: ref #644195
2018-11-27 12:40:22 [INFO    ] Finished processing BAM request
2018-11-27 12:40:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:40:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644195   8 58449 45622029 A  ERROR'
2018-11-27 12:40:22 [DEBUG   ] Got command BAM from MCS: ref #644196
2018-11-27 12:40:22 [INFO    ] Finished processing BAM request
2018-11-27 12:40:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:40:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644196   8 58449 45622033 A  ERROR'
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:40:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:40:22 [DEBUG   ] spawn_process: at 2018-11-27 12:40:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:40:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:40:44 [DEBUG   ] Got command BAM from MCS: ref #644198
2018-11-27 12:40:44 [INFO    ] Finished processing BAM request
2018-11-27 12:40:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:40:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644198   8 58449 45644033 A  ERROR'
2018-11-27 12:40:44 [DEBUG   ] Got command BAM from MCS: ref #644197
2018-11-27 12:40:44 [INFO    ] Finished processing BAM request
2018-11-27 12:40:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:40:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644197   8 58449 45644037 A  ERROR'
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:40:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:40:44 [DEBUG   ] spawn_process: at 2018-11-27 12:40:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:40:49 [DEBUG   ] Got command RPT from MCS: ref #644199
2018-11-27 12:40:49 [DEBUG   ] summary = ERROR
2018-11-27 12:40:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644199  13 58449 45649029 A  ERRORERROR'
2018-11-27 12:40:49 [DEBUG   ] Got command RPT from MCS: ref #644200
2018-11-27 12:40:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:40:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644200  77 58449 45649030 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:40:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:40:58 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:40:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 12:40:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:40:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:40:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:40:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:40:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:40:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:40:59 [DEBUG   ] Finished updating board flags in 0.995 seconds
2018-11-27 12:41:06 [DEBUG   ] Got command BAM from MCS: ref #644217
2018-11-27 12:41:06 [INFO    ] Finished processing BAM request
2018-11-27 12:41:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:41:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644217   8 58449 45666033 A  ERROR'
2018-11-27 12:41:06 [DEBUG   ] Got command BAM from MCS: ref #644216
2018-11-27 12:41:06 [INFO    ] Finished processing BAM request
2018-11-27 12:41:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:41:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644216   8 58449 45666037 A  ERROR'
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:41:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:41:06 [DEBUG   ] spawn_process: at 2018-11-27 12:41:06 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:41:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:41:28 [DEBUG   ] Got command BAM from MCS: ref #644219
2018-11-27 12:41:28 [INFO    ] Finished processing BAM request
2018-11-27 12:41:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:41:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644219   8 58449 45688033 A  ERROR'
2018-11-27 12:41:28 [DEBUG   ] Got command BAM from MCS: ref #644218
2018-11-27 12:41:28 [INFO    ] Finished processing BAM request
2018-11-27 12:41:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:41:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644218   8 58449 45688037 A  ERROR'
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:41:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:41:28 [DEBUG   ] spawn_process: at 2018-11-27 12:41:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:41:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:41:50 [DEBUG   ] Got command BAM from MCS: ref #644221
2018-11-27 12:41:50 [INFO    ] Finished processing BAM request
2018-11-27 12:41:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:41:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644221   8 58449 45710033 A  ERROR'
2018-11-27 12:41:50 [DEBUG   ] Got command BAM from MCS: ref #644220
2018-11-27 12:41:50 [INFO    ] Finished processing BAM request
2018-11-27 12:41:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:41:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644220   8 58449 45710037 A  ERROR'
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:41:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:41:50 [DEBUG   ] spawn_process: at 2018-11-27 12:41:50 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:41:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:41:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:41:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:41:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa33, chip 4
2018-11-27 12:41:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:41:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:41:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:41:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:41:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:41:59 [DEBUG   ] Finished updating board flags in 1.269 seconds
2018-11-27 12:42:12 [DEBUG   ] Got command BAM from MCS: ref #644223
2018-11-27 12:42:12 [INFO    ] Finished processing BAM request
2018-11-27 12:42:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:42:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644223   8 58449 45732029 A  ERROR'
2018-11-27 12:42:12 [DEBUG   ] Got command BAM from MCS: ref #644222
2018-11-27 12:42:12 [INFO    ] Finished processing BAM request
2018-11-27 12:42:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:42:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644222   8 58449 45732033 A  ERROR'
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:42:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:42:12 [DEBUG   ] spawn_process: at 2018-11-27 12:42:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:42:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:42:34 [DEBUG   ] Got command BAM from MCS: ref #644225
2018-11-27 12:42:34 [INFO    ] Finished processing BAM request
2018-11-27 12:42:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:42:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644225   8 58449 45754033 A  ERROR'
2018-11-27 12:42:34 [DEBUG   ] Got command BAM from MCS: ref #644224
2018-11-27 12:42:34 [INFO    ] Finished processing BAM request
2018-11-27 12:42:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:42:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644224   8 58449 45754037 A  ERROR'
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:42:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:42:34 [DEBUG   ] spawn_process: at 2018-11-27 12:42:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:42:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:42:56 [DEBUG   ] Got command BAM from MCS: ref #644229
2018-11-27 12:42:56 [INFO    ] Finished processing BAM request
2018-11-27 12:42:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:42:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644229   8 58449 45776025 A  ERROR'
2018-11-27 12:42:56 [DEBUG   ] Got command BAM from MCS: ref #644228
2018-11-27 12:42:56 [INFO    ] Finished processing BAM request
2018-11-27 12:42:56 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:42:56 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644228   8 58449 45776029 A  ERROR'
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:42:56 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:42:56 [DEBUG   ] spawn_process: at 2018-11-27 12:42:56 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:42:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:42:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 12:42:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 12:42:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:42:59 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:42:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:42:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:42:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:42:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:42:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:42:59 [DEBUG   ] Finished updating board flags in 1.211 seconds
2018-11-27 12:43:05 [DEBUG   ] Got command RPT from MCS: ref #644230
2018-11-27 12:43:05 [DEBUG   ] summary = ERROR
2018-11-27 12:43:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644230  13 58449 45785401 A  ERRORERROR'
2018-11-27 12:43:05 [DEBUG   ] Got command RPT from MCS: ref #644231
2018-11-27 12:43:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:43:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644231  77 58449 45785605 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:43:05 [DEBUG   ] Got command RPT from MCS: ref #644232
2018-11-27 12:43:05 [DEBUG   ] summary = ERROR
2018-11-27 12:43:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644232  13 58449 45785809 A  ERRORERROR'
2018-11-27 12:43:06 [DEBUG   ] Got command RPT from MCS: ref #644234
2018-11-27 12:43:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:43:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644234  77 58449 45786001 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:43:06 [DEBUG   ] Got command RPT from MCS: ref #644236
2018-11-27 12:43:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:43:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644236  12 58449 45786205 A  ERRORB¢ff'
2018-11-27 12:43:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:43:18 [DEBUG   ] Got command BAM from MCS: ref #644245
2018-11-27 12:43:18 [INFO    ] Finished processing BAM request
2018-11-27 12:43:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:43:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644245   8 58449 45798033 A  ERROR'
2018-11-27 12:43:18 [DEBUG   ] Got command BAM from MCS: ref #644244
2018-11-27 12:43:18 [INFO    ] Finished processing BAM request
2018-11-27 12:43:18 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:43:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644244   8 58449 45798037 A  ERROR'
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:43:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:43:18 [DEBUG   ] spawn_process: at 2018-11-27 12:43:18 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:43:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:43:40 [DEBUG   ] Got command BAM from MCS: ref #644246
2018-11-27 12:43:40 [INFO    ] Finished processing BAM request
2018-11-27 12:43:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:43:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644246   8 58449 45820029 A  ERROR'
2018-11-27 12:43:40 [DEBUG   ] Got command BAM from MCS: ref #644247
2018-11-27 12:43:40 [INFO    ] Finished processing BAM request
2018-11-27 12:43:40 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:43:40 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644247   8 58449 45820033 A  ERROR'
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:43:40 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:43:40 [DEBUG   ] spawn_process: at 2018-11-27 12:43:40 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:43:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:43:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 12:43:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 12:43:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:44:00 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:44:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:44:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:44:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:44:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:44:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:44:00 [DEBUG   ] Finished updating board flags in 1.797 seconds
2018-11-27 12:44:02 [DEBUG   ] Got command BAM from MCS: ref #644249
2018-11-27 12:44:02 [INFO    ] Finished processing BAM request
2018-11-27 12:44:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:44:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644249   8 58449 45842025 A  ERROR'
2018-11-27 12:44:02 [DEBUG   ] Got command BAM from MCS: ref #644248
2018-11-27 12:44:02 [INFO    ] Finished processing BAM request
2018-11-27 12:44:02 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:44:02 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644248   8 58449 45842029 A  ERROR'
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:44:02 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:44:02 [DEBUG   ] spawn_process: at 2018-11-27 12:44:02 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:44:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:44:24 [DEBUG   ] Got command BAM from MCS: ref #644251
2018-11-27 12:44:24 [INFO    ] Finished processing BAM request
2018-11-27 12:44:24 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:44:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644251   8 58449 45864029 A  ERROR'
2018-11-27 12:44:24 [DEBUG   ] Got command BAM from MCS: ref #644250
2018-11-27 12:44:24 [INFO    ] Finished processing BAM request
2018-11-27 12:44:24 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:44:24 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644250   8 58449 45864033 A  ERROR'
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:44:24 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:44:24 [DEBUG   ] spawn_process: at 2018-11-27 12:44:24 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:44:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:44:46 [DEBUG   ] Got command BAM from MCS: ref #644252
2018-11-27 12:44:46 [INFO    ] Finished processing BAM request
2018-11-27 12:44:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:44:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644252   8 58449 45886033 A  ERROR'
2018-11-27 12:44:46 [DEBUG   ] Got command BAM from MCS: ref #644253
2018-11-27 12:44:46 [INFO    ] Finished processing BAM request
2018-11-27 12:44:46 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:44:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644253   8 58449 45886037 A  ERROR'
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:44:46 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:44:46 [DEBUG   ] spawn_process: at 2018-11-27 12:44:46 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:44:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:44:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 12:44:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 12:44:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:44:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2018-11-27 12:45:00 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:45:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:45:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:45:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:45:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:45:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:45:00 [DEBUG   ] Finished updating board flags in 1.615 seconds
2018-11-27 12:45:08 [DEBUG   ] Got command BAM from MCS: ref #644254
2018-11-27 12:45:08 [INFO    ] Finished processing BAM request
2018-11-27 12:45:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:45:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644254   8 58449 45908034 A  ERROR'
2018-11-27 12:45:08 [DEBUG   ] Got command BAM from MCS: ref #644255
2018-11-27 12:45:08 [INFO    ] Finished processing BAM request
2018-11-27 12:45:08 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:45:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644255   8 58449 45908038 A  ERROR'
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:45:08 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:45:08 [DEBUG   ] spawn_process: at 2018-11-27 12:45:08 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:45:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:45:30 [DEBUG   ] Got command BAM from MCS: ref #644256
2018-11-27 12:45:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:45:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644256   8 58449 45930030 A  ERROR'
2018-11-27 12:45:30 [DEBUG   ] Got command BAM from MCS: ref #644257
2018-11-27 12:45:30 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:45:30 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644257   8 58449 45930034 A  ERROR'
2018-11-27 12:45:30 [INFO    ] Finished processing BAM request
2018-11-27 12:45:30 [INFO    ] Finished processing BAM request
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:45:30 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:45:30 [DEBUG   ] spawn_process: at 2018-11-27 12:45:30 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:45:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:45:49 [DEBUG   ] Got command RPT from MCS: ref #644258
2018-11-27 12:45:49 [DEBUG   ] summary = ERROR
2018-11-27 12:45:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644258  13 58449 45949051 A  ERRORERROR'
2018-11-27 12:45:49 [DEBUG   ] Got command RPT from MCS: ref #644259
2018-11-27 12:45:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:45:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644259  77 58449 45949052 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:45:52 [DEBUG   ] Got command BAM from MCS: ref #644276
2018-11-27 12:45:52 [INFO    ] Finished processing BAM request
2018-11-27 12:45:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:45:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644276   8 58449 45952030 A  ERROR'
2018-11-27 12:45:52 [DEBUG   ] Got command BAM from MCS: ref #644275
2018-11-27 12:45:52 [INFO    ] Finished processing BAM request
2018-11-27 12:45:52 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:45:52 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644275   8 58449 45952034 A  ERROR'
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:45:52 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:45:52 [DEBUG   ] spawn_process: at 2018-11-27 12:45:52 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:45:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:45:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:45:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 12:45:59 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:45:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:45:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:45:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:45:59 [WARNING ] (13, 1, 'bot')
2018-11-27 12:45:59 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:45:59 [DEBUG   ] Finished updating board flags in 0.962 seconds
2018-11-27 12:46:14 [DEBUG   ] Got command BAM from MCS: ref #644278
2018-11-27 12:46:14 [INFO    ] Finished processing BAM request
2018-11-27 12:46:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:46:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644278   8 58449 45974029 A  ERROR'
2018-11-27 12:46:14 [DEBUG   ] Got command BAM from MCS: ref #644277
2018-11-27 12:46:14 [INFO    ] Finished processing BAM request
2018-11-27 12:46:14 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:46:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644277   8 58449 45974033 A  ERROR'
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:46:14 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:46:14 [DEBUG   ] spawn_process: at 2018-11-27 12:46:14 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:46:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:46:36 [DEBUG   ] Got command BAM from MCS: ref #644283
2018-11-27 12:46:36 [INFO    ] Finished processing BAM request
2018-11-27 12:46:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:46:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644283   8 58449 45996025 A  ERROR'
2018-11-27 12:46:36 [DEBUG   ] Got command BAM from MCS: ref #644284
2018-11-27 12:46:36 [INFO    ] Finished processing BAM request
2018-11-27 12:46:36 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:46:36 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644284   8 58449 45996029 A  ERROR'
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:46:36 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:46:36 [DEBUG   ] spawn_process: at 2018-11-27 12:46:36 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:46:37 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:46:57 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:46:58 [DEBUG   ] Got command BAM from MCS: ref #644285
2018-11-27 12:46:58 [INFO    ] Finished processing BAM request
2018-11-27 12:46:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:46:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644285   8 58449 46018025 A  ERROR'
2018-11-27 12:46:58 [DEBUG   ] Got command BAM from MCS: ref #644286
2018-11-27 12:46:58 [INFO    ] Finished processing BAM request
2018-11-27 12:46:58 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:46:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644286   8 58449 46018029 A  ERROR'
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:46:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:46:58 [DEBUG   ] spawn_process: at 2018-11-27 12:46:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:46:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2018-11-27 12:46:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:47:00 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:47:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:47:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:47:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:47:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:47:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:47:00 [DEBUG   ] Finished updating board flags in 1.358 seconds
2018-11-27 12:47:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 12:47:20 [DEBUG   ] Got command BAM from MCS: ref #644287
2018-11-27 12:47:20 [INFO    ] Finished processing BAM request
2018-11-27 12:47:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:47:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644287   8 58449 46040033 A  ERROR'
2018-11-27 12:47:20 [DEBUG   ] Got command BAM from MCS: ref #644288
2018-11-27 12:47:20 [INFO    ] Finished processing BAM request
2018-11-27 12:47:20 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:47:20 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644288   8 58449 46040037 A  ERROR'
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:47:20 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:47:20 [DEBUG   ] spawn_process: at 2018-11-27 12:47:20 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:47:37 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 12:47:42 [DEBUG   ] Got command BAM from MCS: ref #644289
2018-11-27 12:47:42 [INFO    ] Finished processing BAM request
2018-11-27 12:47:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:47:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644289   8 58449 46062025 A  ERROR'
2018-11-27 12:47:42 [DEBUG   ] Got command BAM from MCS: ref #644290
2018-11-27 12:47:42 [INFO    ] Finished processing BAM request
2018-11-27 12:47:42 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:47:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644290   8 58449 46062029 A  ERROR'
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:47:42 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:47:42 [DEBUG   ] spawn_process: at 2018-11-27 12:47:42 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:47:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:47:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 12:47:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 12:47:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:47:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 4
2018-11-27 12:48:00 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:48:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 12:48:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:48:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:48:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:48:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:48:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:48:00 [DEBUG   ] Finished updating board flags in 1.217 seconds
2018-11-27 12:48:04 [DEBUG   ] Got command BAM from MCS: ref #644292
2018-11-27 12:48:04 [INFO    ] Finished processing BAM request
2018-11-27 12:48:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:48:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644292   8 58449 46084033 A  ERROR'
2018-11-27 12:48:04 [DEBUG   ] Got command BAM from MCS: ref #644291
2018-11-27 12:48:04 [INFO    ] Finished processing BAM request
2018-11-27 12:48:04 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:48:04 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644291   8 58449 46084037 A  ERROR'
2018-11-27 12:48:04 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:48:04 [DEBUG   ] spawn_process: at 2018-11-27 12:48:04 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:48:05 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:48:05 [DEBUG   ] spawn_process: at 2018-11-27 12:48:05 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:48:05 [DEBUG   ] Got command RPT from MCS: ref #644293
2018-11-27 12:48:05 [DEBUG   ] summary = ERROR
2018-11-27 12:48:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644293  13 58449 46085489 A  ERRORERROR'
2018-11-27 12:48:05 [DEBUG   ] Got command RPT from MCS: ref #644294
2018-11-27 12:48:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:48:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644294  77 58449 46085693 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:48:05 [DEBUG   ] Got command RPT from MCS: ref #644295
2018-11-27 12:48:05 [DEBUG   ] summary = ERROR
2018-11-27 12:48:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644295  13 58449 46085897 A  ERRORERROR'
2018-11-27 12:48:06 [DEBUG   ] Got command RPT from MCS: ref #644296
2018-11-27 12:48:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:48:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644296  77 58449 46086089 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:48:06 [DEBUG   ] Got command RPT from MCS: ref #644297
2018-11-27 12:48:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:48:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644297  12 58449 46086293 A  ERRORB£ff'
2018-11-27 12:48:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 12:48:26 [DEBUG   ] Got command BAM from MCS: ref #644298
2018-11-27 12:48:26 [INFO    ] Finished processing BAM request
2018-11-27 12:48:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:48:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644298   8 58449 46106025 A  ERROR'
2018-11-27 12:48:26 [DEBUG   ] Got command BAM from MCS: ref #644299
2018-11-27 12:48:26 [INFO    ] Finished processing BAM request
2018-11-27 12:48:26 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:48:26 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644299   8 58449 46106029 A  ERROR'
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2018-11-27 12:48:26 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2018-11-27 12:48:26 [DEBUG   ] spawn_process: at 2018-11-27 12:48:26 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2018-11-27 12:48:37 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2018-11-27 12:48:48 [DEBUG   ] Got command BAM from MCS: ref #644300
2018-11-27 12:48:48 [INFO    ] Finished processing BAM request
2018-11-27 12:48:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:48:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644300   8 58449 46128029 A  ERROR'
2018-11-27 12:48:48 [DEBUG   ] Got command BAM from MCS: ref #644301
2018-11-27 12:48:48 [INFO    ] Finished processing BAM request
2018-11-27 12:48:48 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:48:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644301   8 58449 46128033 A  ERROR'
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:48:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:48:48 [DEBUG   ] spawn_process: at 2018-11-27 12:48:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:48:57 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:48:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2018-11-27 12:48:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:49:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 1
2018-11-27 12:49:00 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:49:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 12:49:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:49:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:49:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:49:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:49:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:49:00 [DEBUG   ] Finished updating board flags in 1.204 seconds
2018-11-27 12:49:10 [DEBUG   ] Got command BAM from MCS: ref #644303
2018-11-27 12:49:10 [INFO    ] Finished processing BAM request
2018-11-27 12:49:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:49:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644303   8 58449 46150033 A  ERROR'
2018-11-27 12:49:10 [DEBUG   ] Got command BAM from MCS: ref #644302
2018-11-27 12:49:10 [INFO    ] Finished processing BAM request
2018-11-27 12:49:10 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:49:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644302   8 58449 46150037 A  ERROR'
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:49:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:49:10 [DEBUG   ] spawn_process: at 2018-11-27 12:49:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:49:17 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:49:32 [DEBUG   ] Got command BAM from MCS: ref #644308
2018-11-27 12:49:32 [INFO    ] Finished processing BAM request
2018-11-27 12:49:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:49:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644308   8 58449 46172025 A  ERROR'
2018-11-27 12:49:32 [DEBUG   ] Got command BAM from MCS: ref #644309
2018-11-27 12:49:32 [INFO    ] Finished processing BAM request
2018-11-27 12:49:32 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:49:32 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644309   8 58449 46172029 A  ERROR'
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2018-11-27 12:49:32 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2018-11-27 12:49:32 [DEBUG   ] spawn_process: at 2018-11-27 12:49:32 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2018-11-27 12:49:37 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:49:54 [DEBUG   ] Got command BAM from MCS: ref #644310
2018-11-27 12:49:54 [INFO    ] Finished processing BAM request
2018-11-27 12:49:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:49:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644310   8 58449 46194029 A  ERROR'
2018-11-27 12:49:54 [DEBUG   ] Got command BAM from MCS: ref #644311
2018-11-27 12:49:54 [INFO    ] Finished processing BAM request
2018-11-27 12:49:54 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:49:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644311   8 58449 46194033 A  ERROR'
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:49:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:49:54 [DEBUG   ] spawn_process: at 2018-11-27 12:49:54 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:49:57 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2018-11-27 12:49:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 2
2018-11-27 12:49:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:50:00 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:50:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 12:50:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:50:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:50:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:50:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:50:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:50:00 [DEBUG   ] Finished updating board flags in 1.183 seconds
2018-11-27 12:50:16 [DEBUG   ] Got command BAM from MCS: ref #644312
2018-11-27 12:50:16 [INFO    ] Finished processing BAM request
2018-11-27 12:50:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:50:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644312   8 58449 46216029 A  ERROR'
2018-11-27 12:50:16 [DEBUG   ] Got command BAM from MCS: ref #644313
2018-11-27 12:50:16 [INFO    ] Finished processing BAM request
2018-11-27 12:50:16 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:50:16 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644313   8 58449 46216033 A  ERROR'
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:50:16 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:50:16 [DEBUG   ] spawn_process: at 2018-11-27 12:50:16 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:50:17 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2018-11-27 12:50:38 [DEBUG   ] Got command BAM from MCS: ref #644314
2018-11-27 12:50:38 [INFO    ] Finished processing BAM request
2018-11-27 12:50:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:50:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644314   8 58449 46238025 A  ERROR'
2018-11-27 12:50:38 [DEBUG   ] Got command BAM from MCS: ref #644315
2018-11-27 12:50:38 [INFO    ] Finished processing BAM request
2018-11-27 12:50:38 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:50:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644315   8 58449 46238029 A  ERROR'
2018-11-27 12:50:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:50:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:50:38 [DEBUG   ] spawn_process: at 2018-11-27 12:50:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:50:49 [DEBUG   ] Got command RPT from MCS: ref #644316
2018-11-27 12:50:49 [DEBUG   ] summary = ERROR
2018-11-27 12:50:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644316  13 58449 46249033 A  ERRORERROR'
2018-11-27 12:50:49 [DEBUG   ] Got command RPT from MCS: ref #644317
2018-11-27 12:50:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:50:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644317  77 58449 46249034 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:50:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 12:50:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 3
2018-11-27 12:50:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:51:00 [DEBUG   ] Got command BAM from MCS: ref #644334
2018-11-27 12:51:00 [INFO    ] Finished processing BAM request
2018-11-27 12:51:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:51:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644334   8 58449 46260033 A  ERROR'
2018-11-27 12:51:00 [DEBUG   ] Got command BAM from MCS: ref #644333
2018-11-27 12:51:00 [INFO    ] Finished processing BAM request
2018-11-27 12:51:00 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:51:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644333   8 58449 46260041 A  ERROR'
2018-11-27 12:51:00 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:51:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 2
2018-11-27 12:51:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:51:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:51:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:51:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:51:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:51:00 [DEBUG   ] Finished updating board flags in 1.007 seconds
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:51:00 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:51:00 [DEBUG   ] spawn_process: at 2018-11-27 12:51:00 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:51:18 [DEBUG   ] Finished updating ant stats in 1.194 seconds
2018-11-27 12:51:22 [DEBUG   ] Got command BAM from MCS: ref #644336
2018-11-27 12:51:22 [INFO    ] Finished processing BAM request
2018-11-27 12:51:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:51:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644336   8 58449 46282029 A  ERROR'
2018-11-27 12:51:22 [DEBUG   ] Got command BAM from MCS: ref #644335
2018-11-27 12:51:22 [INFO    ] Finished processing BAM request
2018-11-27 12:51:22 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:51:22 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644335   8 58449 46282033 A  ERROR'
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:51:22 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:51:22 [DEBUG   ] spawn_process: at 2018-11-27 12:51:22 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:51:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:51:44 [DEBUG   ] Got command BAM from MCS: ref #644338
2018-11-27 12:51:44 [INFO    ] Finished processing BAM request
2018-11-27 12:51:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:51:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644338   8 58449 46304033 A  ERROR'
2018-11-27 12:51:44 [DEBUG   ] Got command BAM from MCS: ref #644337
2018-11-27 12:51:44 [INFO    ] Finished processing BAM request
2018-11-27 12:51:44 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:51:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644337   8 58449 46304037 A  ERROR'
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:51:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:51:44 [DEBUG   ] spawn_process: at 2018-11-27 12:51:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:51:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:51:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 12:51:59 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:52:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 12:52:00 [WARNING ] MonitorFlags: Temperature 81.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:52:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:52:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:52:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:52:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:52:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:52:00 [DEBUG   ] Finished updating board flags in 0.998 seconds
2018-11-27 12:52:06 [DEBUG   ] Got command BAM from MCS: ref #644339
2018-11-27 12:52:06 [INFO    ] Finished processing BAM request
2018-11-27 12:52:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644339   8 58449 46326033 A  ERROR'
2018-11-27 12:52:06 [DEBUG   ] Got command BAM from MCS: ref #644340
2018-11-27 12:52:06 [INFO    ] Finished processing BAM request
2018-11-27 12:52:06 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644340   8 58449 46326037 A  ERROR'
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2018-11-27 12:52:06 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2018-11-27 12:52:06 [DEBUG   ] spawn_process: at 2018-11-27 12:52:06 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2018-11-27 12:52:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:52:28 [DEBUG   ] Got command BAM from MCS: ref #644346
2018-11-27 12:52:28 [INFO    ] Finished processing BAM request
2018-11-27 12:52:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:52:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644346   8 58449 46348029 A  ERROR'
2018-11-27 12:52:28 [DEBUG   ] Got command BAM from MCS: ref #644345
2018-11-27 12:52:28 [INFO    ] Finished processing BAM request
2018-11-27 12:52:28 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:52:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644345   8 58449 46348033 A  ERROR'
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:52:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:52:28 [DEBUG   ] spawn_process: at 2018-11-27 12:52:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:52:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:52:50 [DEBUG   ] Got command BAM from MCS: ref #644349
2018-11-27 12:52:50 [INFO    ] Finished processing BAM request
2018-11-27 12:52:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:52:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644349   8 58449 46370165 A  ERROR'
2018-11-27 12:52:50 [DEBUG   ] Got command BAM from MCS: ref #644350
2018-11-27 12:52:50 [INFO    ] Finished processing BAM request
2018-11-27 12:52:50 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:52:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644350   8 58449 46370169 A  ERROR'
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:52:50 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:52:50 [DEBUG   ] spawn_process: at 2018-11-27 12:52:50 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:52:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:53:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 12:53:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2018-11-27 12:53:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:53:00 [WARNING ] MonitorFlags: Temperature 81.70 > 80 on hostname lwa21, chip 2
2018-11-27 12:53:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:53:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:53:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:53:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:53:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:53:00 [DEBUG   ] Finished updating board flags in 1.268 seconds
2018-11-27 12:53:05 [DEBUG   ] Got command RPT from MCS: ref #644351
2018-11-27 12:53:05 [DEBUG   ] summary = ERROR
2018-11-27 12:53:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644351  13 58449 46385573 A  ERRORERROR'
2018-11-27 12:53:05 [DEBUG   ] Got command RPT from MCS: ref #644352
2018-11-27 12:53:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:53:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644352  77 58449 46385777 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:53:05 [DEBUG   ] Got command RPT from MCS: ref #644353
2018-11-27 12:53:05 [DEBUG   ] summary = ERROR
2018-11-27 12:53:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644353  13 58449 46385981 A  ERRORERROR'
2018-11-27 12:53:06 [DEBUG   ] Got command RPT from MCS: ref #644355
2018-11-27 12:53:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:53:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644355  77 58449 46386185 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:53:06 [DEBUG   ] Got command RPT from MCS: ref #644356
2018-11-27 12:53:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:53:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644356  12 58449 46386377 A  ERRORB£ff'
2018-11-27 12:53:12 [DEBUG   ] Got command BAM from MCS: ref #644362
2018-11-27 12:53:12 [INFO    ] Finished processing BAM request
2018-11-27 12:53:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:53:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644362   8 58449 46392033 A  ERROR'
2018-11-27 12:53:12 [DEBUG   ] Got command BAM from MCS: ref #644361
2018-11-27 12:53:12 [INFO    ] Finished processing BAM request
2018-11-27 12:53:12 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:53:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644361   8 58449 46392037 A  ERROR'
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2018-11-27 12:53:12 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2018-11-27 12:53:12 [DEBUG   ] spawn_process: at 2018-11-27 12:53:12 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2018-11-27 12:53:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:53:34 [DEBUG   ] Got command BAM from MCS: ref #644363
2018-11-27 12:53:34 [INFO    ] Finished processing BAM request
2018-11-27 12:53:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:53:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644363   8 58449 46414033 A  ERROR'
2018-11-27 12:53:34 [DEBUG   ] Got command BAM from MCS: ref #644364
2018-11-27 12:53:34 [INFO    ] Finished processing BAM request
2018-11-27 12:53:34 [DEBUG   ] bam = exited with status True, code 0x00
2018-11-27 12:53:34 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   644364   8 58449 46414037 A  ERROR'
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:53:34 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:53:34 [DEBUG   ] spawn_process: at 2018-11-27 12:53:34 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:53:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:53:54 [DEBUG   ] Got command STP from MCS: ref #644366
2018-11-27 12:53:54 [INFO    ] Finished processing BAM request
2018-11-27 12:53:54 [INFO    ] Shutdown the adder chain on beam #2
2018-11-27 12:53:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   644366   8 58449 46434033 A  ERROR'
2018-11-27 12:53:54 [DEBUG   ] Got command STP from MCS: ref #644365
2018-11-27 12:53:54 [INFO    ] Finished processing BAM request
2018-11-27 12:53:54 [INFO    ] Shutdown the adder chain on beam #3
2018-11-27 12:53:54 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_STP   644365   8 58449 46434037 A  ERROR'
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2018-11-27 12:53:54 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2018-11-27 12:53:54 [DEBUG   ] spawn_process: at 2018-11-27 12:53:54 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2018-11-27 12:53:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:54:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:54:01 [WARNING ] MonitorFlags: Temperature 80.20 > 80 on hostname lwa21, chip 2
2018-11-27 12:54:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:54:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:54:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:54:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:54:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:54:01 [DEBUG   ] Finished updating board flags in 1.757 seconds
2018-11-27 12:54:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:54:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:54:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:55:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:55:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:55:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:55:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:55:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:55:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:55:00 [DEBUG   ] Finished updating board flags in 1.049 seconds
2018-11-27 12:55:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:55:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:55:49 [DEBUG   ] Got command RPT from MCS: ref #644371
2018-11-27 12:55:49 [DEBUG   ] summary = ERROR
2018-11-27 12:55:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644371  13 58449 46549037 A  ERRORERROR'
2018-11-27 12:55:49 [DEBUG   ] Got command RPT from MCS: ref #644372
2018-11-27 12:55:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:55:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644372  77 58449 46549038 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:55:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:56:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:56:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:56:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:56:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:56:00 [WARNING ] (13, 1, 'bot')
2018-11-27 12:56:00 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:56:00 [DEBUG   ] Finished updating board flags in 0.989 seconds
2018-11-27 12:56:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:56:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:56:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:57:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:57:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 12:57:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:57:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:57:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:57:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:57:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:57:01 [DEBUG   ] Finished updating board flags in 1.419 seconds
2018-11-27 12:57:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:57:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:57:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:58:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:58:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:58:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:58:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:58:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:58:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:58:01 [DEBUG   ] Finished updating board flags in 1.335 seconds
2018-11-27 12:58:05 [DEBUG   ] Got command RPT from MCS: ref #644388
2018-11-27 12:58:05 [DEBUG   ] summary = ERROR
2018-11-27 12:58:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644388  13 58449 46685661 A  ERRORERROR'
2018-11-27 12:58:05 [DEBUG   ] Got command RPT from MCS: ref #644389
2018-11-27 12:58:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:58:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644389  77 58449 46685865 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:58:06 [DEBUG   ] Got command RPT from MCS: ref #644390
2018-11-27 12:58:06 [DEBUG   ] summary = ERROR
2018-11-27 12:58:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644390  13 58449 46686069 A  ERRORERROR'
2018-11-27 12:58:06 [DEBUG   ] Got command RPT from MCS: ref #644391
2018-11-27 12:58:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 12:58:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644391  77 58449 46686273 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 12:58:06 [DEBUG   ] Got command RPT from MCS: ref #644392
2018-11-27 12:58:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 12:58:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644392  12 58449 46686465 A  ERRORB™™š'
2018-11-27 12:58:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:58:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:58:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:59:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 12:59:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 12:59:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 12:59:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:59:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:59:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:59:01 [WARNING ] (13, 1, 'bot')
2018-11-27 12:59:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 12:59:01 [DEBUG   ] Finished updating board flags in 1.203 seconds
2018-11-27 12:59:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:59:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 12:59:58 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 13:00:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:00:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:00:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:00:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:00:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:00:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:00:01 [DEBUG   ] Finished updating board flags in 1.193 seconds
2018-11-27 13:00:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 13:00:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 13:00:49 [DEBUG   ] Got command RPT from MCS: ref #644397
2018-11-27 13:00:49 [DEBUG   ] summary = ERROR
2018-11-27 13:00:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644397  13 58449 46849033 A  ERRORERROR'
2018-11-27 13:00:49 [DEBUG   ] Got command RPT from MCS: ref #644398
2018-11-27 13:00:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:00:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644398  77 58449 46849034 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:00:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 13:01:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:01:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:01:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:01:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:01:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:01:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:01:01 [DEBUG   ] Finished updating board flags in 1.007 seconds
2018-11-27 13:01:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 13:01:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 13:01:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 13:02:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:02:01 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 13:02:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:02:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:02:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:02:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:02:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:02:01 [DEBUG   ] Finished updating board flags in 1.038 seconds
2018-11-27 13:02:18 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2018-11-27 13:02:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 13:02:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2018-11-27 13:03:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:03:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:03:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:03:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:03:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:03:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:03:02 [DEBUG   ] Finished updating board flags in 1.534 seconds
2018-11-27 13:03:05 [DEBUG   ] Got command RPT from MCS: ref #644420
2018-11-27 13:03:05 [DEBUG   ] summary = ERROR
2018-11-27 13:03:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644420  13 58449 46985745 A  ERRORERROR'
2018-11-27 13:03:05 [DEBUG   ] Got command RPT from MCS: ref #644421
2018-11-27 13:03:05 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:03:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644421  77 58449 46985949 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:03:06 [DEBUG   ] Got command RPT from MCS: ref #644422
2018-11-27 13:03:06 [DEBUG   ] summary = ERROR
2018-11-27 13:03:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644422  13 58449 46986153 A  ERRORERROR'
2018-11-27 13:03:06 [DEBUG   ] Got command RPT from MCS: ref #644424
2018-11-27 13:03:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:03:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644424  77 58449 46986345 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:03:06 [DEBUG   ] Got command RPT from MCS: ref #644426
2018-11-27 13:03:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 13:03:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644426  12 58449 46986549 A  ERRORB˜™š'
2018-11-27 13:03:18 [DEBUG   ] Finished updating ant stats in 0.990 seconds
2018-11-27 13:03:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:03:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:04:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:04:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:04:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:04:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:04:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:04:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:04:01 [DEBUG   ] Finished updating board flags in 1.272 seconds
2018-11-27 13:04:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:04:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:04:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:05:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 13:05:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:05:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:05:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:05:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:05:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:05:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:05:02 [DEBUG   ] Finished updating board flags in 1.768 seconds
2018-11-27 13:05:18 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 13:05:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:05:49 [DEBUG   ] Got command RPT from MCS: ref #644434
2018-11-27 13:05:49 [DEBUG   ] summary = ERROR
2018-11-27 13:05:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644434  13 58449 47149041 A  ERRORERROR'
2018-11-27 13:05:49 [DEBUG   ] Got command RPT from MCS: ref #644435
2018-11-27 13:05:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:05:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644435  77 58449 47149042 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:05:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:06:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 4
2018-11-27 13:06:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:06:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa25, chip 4
2018-11-27 13:06:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:06:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:06:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:06:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:06:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:06:02 [DEBUG   ] Finished updating board flags in 1.616 seconds
2018-11-27 13:06:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:06:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:06:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:07:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:07:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:07:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:07:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:07:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:07:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:07:01 [DEBUG   ] Finished updating board flags in 1.039 seconds
2018-11-27 13:07:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:07:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:07:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:08:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:08:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 13:08:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:08:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:08:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:08:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:08:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:08:01 [DEBUG   ] Finished updating board flags in 1.049 seconds
2018-11-27 13:08:05 [DEBUG   ] Got command RPT from MCS: ref #644455
2018-11-27 13:08:05 [DEBUG   ] summary = ERROR
2018-11-27 13:08:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644455  13 58449 47285833 A  ERRORERROR'
2018-11-27 13:08:06 [DEBUG   ] Got command RPT from MCS: ref #644456
2018-11-27 13:08:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:08:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644456  77 58449 47286037 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:08:06 [DEBUG   ] Got command RPT from MCS: ref #644457
2018-11-27 13:08:06 [DEBUG   ] summary = ERROR
2018-11-27 13:08:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644457  13 58449 47286241 A  ERRORERROR'
2018-11-27 13:08:06 [DEBUG   ] Got command RPT from MCS: ref #644458
2018-11-27 13:08:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:08:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644458  77 58449 47286445 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:08:06 [DEBUG   ] Got command RPT from MCS: ref #644459
2018-11-27 13:08:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 13:08:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644459  12 58449 47286637 A  ERRORB™™š'
2018-11-27 13:08:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:08:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:08:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:09:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:09:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:09:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:09:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:09:01 [WARNING ] (13, 1, 'bot')
2018-11-27 13:09:01 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:09:01 [DEBUG   ] Finished updating board flags in 1.033 seconds
2018-11-27 13:09:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:09:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:09:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:10:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:10:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:10:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:10:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:10:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:10:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:10:02 [DEBUG   ] Finished updating board flags in 1.545 seconds
2018-11-27 13:10:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:10:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:10:49 [DEBUG   ] Got command RPT from MCS: ref #644464
2018-11-27 13:10:49 [DEBUG   ] summary = ERROR
2018-11-27 13:10:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644464  13 58449 47449041 A  ERRORERROR'
2018-11-27 13:10:49 [DEBUG   ] Got command RPT from MCS: ref #644465
2018-11-27 13:10:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:10:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644465  77 58449 47449042 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:10:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:11:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:11:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:11:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:11:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:11:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:11:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:11:02 [DEBUG   ] Finished updating board flags in 1.413 seconds
2018-11-27 13:11:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:11:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:11:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:12:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:12:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:12:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:12:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:12:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:12:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:12:02 [DEBUG   ] Finished updating board flags in 1.322 seconds
2018-11-27 13:12:18 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 13:12:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:12:58 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2018-11-27 13:13:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:13:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:13:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:13:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:13:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:13:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:13:02 [DEBUG   ] Finished updating board flags in 1.217 seconds
2018-11-27 13:13:05 [DEBUG   ] Got command RPT from MCS: ref #644483
2018-11-27 13:13:05 [DEBUG   ] summary = ERROR
2018-11-27 13:13:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644483  13 58449 47585921 A  ERRORERROR'
2018-11-27 13:13:06 [DEBUG   ] Got command RPT from MCS: ref #644484
2018-11-27 13:13:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:13:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644484  77 58449 47586125 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:13:06 [DEBUG   ] Got command RPT from MCS: ref #644485
2018-11-27 13:13:06 [DEBUG   ] summary = ERROR
2018-11-27 13:13:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644485  13 58449 47586317 A  ERRORERROR'
2018-11-27 13:13:06 [DEBUG   ] Got command RPT from MCS: ref #644487
2018-11-27 13:13:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:13:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644487  77 58449 47586521 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:13:06 [DEBUG   ] Got command RPT from MCS: ref #644489
2018-11-27 13:13:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 13:13:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644489  12 58449 47586725 A  ERRORB˜™š'
2018-11-27 13:13:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:13:38 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:13:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2018-11-27 13:14:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:14:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:14:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:14:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:14:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:14:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:14:02 [DEBUG   ] Finished updating board flags in 1.172 seconds
2018-11-27 13:14:18 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2018-11-27 13:14:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 13:14:58 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 13:15:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:15:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:15:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:15:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:15:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:15:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:15:02 [DEBUG   ] Finished updating board flags in 1.024 seconds
2018-11-27 13:15:18 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2018-11-27 13:15:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 13:15:49 [DEBUG   ] Got command RPT from MCS: ref #644497
2018-11-27 13:15:49 [DEBUG   ] summary = ERROR
2018-11-27 13:15:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644497  13 58449 47749041 A  ERRORERROR'
2018-11-27 13:15:49 [DEBUG   ] Got command RPT from MCS: ref #644498
2018-11-27 13:15:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:15:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644498  77 58449 47749042 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:15:58 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 13:16:01 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:16:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:16:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:16:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:16:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:16:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:16:02 [DEBUG   ] Finished updating board flags in 0.990 seconds
2018-11-27 13:16:18 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 13:16:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 13:16:58 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 13:17:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:17:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:17:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:17:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:17:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:17:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:17:03 [DEBUG   ] Finished updating board flags in 1.851 seconds
2018-11-27 13:17:18 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 13:17:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 13:17:58 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2018-11-27 13:18:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:18:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:18:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:18:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:18:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:18:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:18:02 [DEBUG   ] Finished updating board flags in 1.236 seconds
2018-11-27 13:18:06 [DEBUG   ] Got command RPT from MCS: ref #644518
2018-11-27 13:18:06 [DEBUG   ] summary = ERROR
2018-11-27 13:18:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644518  13 58449 47886001 A  ERRORERROR'
2018-11-27 13:18:06 [DEBUG   ] Got command RPT from MCS: ref #644519
2018-11-27 13:18:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:18:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644519  77 58449 47886205 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:18:06 [DEBUG   ] Got command RPT from MCS: ref #644520
2018-11-27 13:18:06 [DEBUG   ] summary = ERROR
2018-11-27 13:18:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644520  13 58449 47886409 A  ERRORERROR'
2018-11-27 13:18:06 [DEBUG   ] Got command RPT from MCS: ref #644521
2018-11-27 13:18:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:18:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644521  77 58449 47886613 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:18:06 [DEBUG   ] Got command RPT from MCS: ref #644522
2018-11-27 13:18:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 13:18:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644522  12 58449 47886805 A  ERRORB˜™š'
2018-11-27 13:18:12 [DEBUG   ] Got command TBW from MCS: ref #644524
2018-11-27 13:18:12 [DEBUG   ] tbw = exited with status True, code 0x00
2018-11-27 13:18:12 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBW   644524   8 58449 47892017 A  ERROR'
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 pkill tbn
2018-11-27 13:18:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 pkill tbn
2018-11-27 13:18:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 pkill tbn
2018-11-27 13:18:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 pkill tbn
2018-11-27 13:18:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 pkill tbn
2018-11-27 13:18:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 pkill tbn
2018-11-27 13:18:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 pkill tbn
2018-11-27 13:18:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 pkill tbn
2018-11-27 13:18:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 pkill tbn
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa27', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '1', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa24', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '2', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa23', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '3', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa18', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '4', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa10', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '5', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa32', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '6', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa11', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '7', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa30', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '8', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa12', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '9', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa8', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '10', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa13', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '11', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa26', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '12', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa14', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '13', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa22', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '14', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa20', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '15', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa1', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '16', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa5', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '17', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa25', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '18', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa35', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '19', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa19', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '20', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa21', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '21', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa6', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '22', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa7', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '23', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa33', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '24', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa15', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '25', '12000000']
2018-11-27 13:18:14 [DEBUG   ] spawn_process: at 2018-11-27 13:18:14 running ['/usr/local/bin/lwa28', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '26', '12000000']
2018-11-27 13:18:14 [INFO    ] Finished spawning TBW
2018-11-27 13:19:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:19:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:19:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:19:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:19:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:19:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:19:03 [DEBUG   ] Finished updating board flags in 1.820 seconds
2018-11-27 13:20:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:20:02 [INFO    ] __waitForTBW - TBW finished in 108.678 s
2018-11-27 13:20:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 2
2018-11-27 13:20:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:20:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:20:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:20:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:20:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:20:03 [DEBUG   ] Finished updating board flags in 1.720 seconds
2018-11-27 13:20:03 [INFO    ] Started antennas statistics background thread
2018-11-27 13:20:04 [DEBUG   ] Finished updating ant stats in 1.297 seconds
2018-11-27 13:20:24 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2018-11-27 13:20:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:20:49 [DEBUG   ] Got command RPT from MCS: ref #644536
2018-11-27 13:20:49 [DEBUG   ] summary = ERROR
2018-11-27 13:20:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644536  13 58449 48049041 A  ERRORERROR'
2018-11-27 13:20:49 [DEBUG   ] Got command RPT from MCS: ref #644537
2018-11-27 13:20:49 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:20:49 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644537  77 58449 48049042 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:21:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:21:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:21:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:21:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:21:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:21:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:21:02 [DEBUG   ] Finished updating board flags in 1.018 seconds
2018-11-27 13:21:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:21:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:21:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:22:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:22:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:22:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:22:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:22:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:22:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:22:02 [DEBUG   ] Finished updating board flags in 1.030 seconds
2018-11-27 13:22:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:22:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:22:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:23:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2018-11-27 13:23:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:23:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:23:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:23:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:23:02 [WARNING ] (13, 1, 'bot')
2018-11-27 13:23:02 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:23:02 [DEBUG   ] Finished updating board flags in 0.974 seconds
2018-11-27 13:23:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:23:06 [DEBUG   ] Got command RPT from MCS: ref #644559
2018-11-27 13:23:06 [DEBUG   ] summary = ERROR
2018-11-27 13:23:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644559  13 58449 48186097 A  ERRORERROR'
2018-11-27 13:23:06 [DEBUG   ] Got command RPT from MCS: ref #644560
2018-11-27 13:23:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:23:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644560  77 58449 48186289 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:23:06 [DEBUG   ] Got command RPT from MCS: ref #644561
2018-11-27 13:23:06 [DEBUG   ] summary = ERROR
2018-11-27 13:23:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644561  13 58449 48186493 A  ERRORERROR'
2018-11-27 13:23:06 [DEBUG   ] Got command RPT from MCS: ref #644563
2018-11-27 13:23:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2018-11-27 13:23:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644563  77 58449 48186697 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2018-11-27 13:23:06 [DEBUG   ] Got command RPT from MCS: ref #644564
2018-11-27 13:23:06 [DEBUG   ] GLOBAL_TEMP_MAX = exited with status True
2018-11-27 13:23:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   644564  12 58449 48186901 A  ERRORB˜™š'
2018-11-27 13:23:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:23:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:24:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa26, chip 1, bottom port
2018-11-27 13:24:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:24:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:24:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:24:03 [WARNING ] (13, 1, 'bot')
2018-11-27 13:24:03 [CRITICAL] MGT underflow detected on beam(s) 1
2018-11-27 13:24:03 [DEBUG   ] Finished updating board flags in 1.418 seconds
2018-11-27 13:24:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:24:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2018-11-27 13:24:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
