When receiving DWA from a peer, the timeout event and the DWA event are coming almost at the same time, which causes the peer to be disconnected. Two threads are doing it at the same time, which we have a race condition. You can see the log that 2 threads are executing the Event, and the Timeout.
2016-11-02 23:50:46,475 DEBUG [PeerFSMImpl] (Thread-14) Handling event with type [DWA_EVENT]
2016-11-02 23:50:46,475 DEBUG [PeerFSMImpl] (Thread-14) Not performing validation to message since validator is DISABLED.
2016-11-02 23:50:46,475 DEBUG [PeerFSMImpl] (Thread-14) Placing event [Event{name:DWA_EVENT, key:aaa://10.225.20.206:16018, object:MessageImpl{commandCode=280, flags=0}}] into linked blocking queue with remaining capacity: [10000].
2016-11-02 23:50:46,475 DEBUG [PeerFSMImpl] (FSM-SPeer{Uri=aaa://10.225.20.206; State=null; con=null; incConnull }_2-1) SPeer{Uri=aaa://10.225.20.206; State=OKAY; con=org.jdiameter.server.impl.io.sctp.SCTPServerConnection@162db65b; incCon{} } FSM switch state: OKAY -> SUSPECT
2016-11-02 23:50:46,477 DEBUG [PeerFSMImpl] (FSM-SPeer{Uri=aaa://10.225.20.206; State=null; con=null; incConnull }_10-9) Got Event [Event{name:DWA_EVENT, key:aaa://10.225.20.206:16018, object:MessageImpl{commandCode=280, flags=0}}] from Queue
2016-11-02 23:50:46,478 DEBUG [PeerFSMImpl] (FSM-SPeer{Uri=aaa://10.225.20.206; State=null; con=null; incConnull }_10-9) Process event [Event{name:DWA_EVENT, key:aaa://10.225.20.206:16018, object:MessageImpl{commandCode=280, flags=0}}]. Peer State is [OKAY]
2016-11-02 23:50:55,466 DEBUG [TimerTask] (SLEE-TimerFacility-thread-3) Task with id -6ee284d6:158250d0338:-7fbb is recurring, not removing it locally nor in the cluster
2016-11-02 23:50:55,466 DEBUG [TimerTask] (SLEE-TimerFacility-thread-3) Firing Timer with id -6ee284d6:158250d0338:-7fbb
2016-11-02 23:50:55,466 DEBUG [TimerFacilityTimerTask] (SLEE-TimerFacility-thread-3) Executing task with timer ID -6ee284d6:158250d0338:-7fbb
2016-11-02 23:50:55,466 DEBUG [TimerFacilityTimerTask] (SLEE-TimerFacility-thread-3) Delay till execution is 1
2016-11-02 23:50:55,466 DEBUG [TimerFacilityTimerTask] (SLEE-TimerFacility-thread-3) Remaining executions:2147483647
2016-11-02 23:51:15,519 DEBUG [PeerFSMImpl] (FSM-SPeer{Uri=aaa://10.225.20.206; State=null; con=null; incConnull }_8-7) Sending timeout event
2016-11-02 23:51:15,520 DEBUG [PeerFSMImpl] (FSM-SPeer{Uri=aaa://10.225.20.206; State=null; con=null; incConnull }_8-7) Handling event with type [TIMEOUT_EVENT]
2016-11-02 23:51:15,520 DEBUG [PeerFSMImpl] (FSM-SPeer{Uri=aaa://10.225.20.206; State=null; con=null; incConnull }_8-7) Not performing validation to message since validator is DISABLED.
2016-11-02 23:51:15,520 DEBUG [PeerFSMImpl] (FSM-SPeer{Uri=aaa://10.225.20.206; State=null; con=null; incConnull }_8-7) Placing event [Event{name:TIMEOUT_EVENT, key:null, object:null}] into linked blocking queue with remaining capacity: [10000].
2016-11-02 23:51:15,520 DEBUG [PeerFSMImpl] (FSM-SPeer{Uri=aaa://10.225.20.206; State=null; con=null; incConnull }_8-7) Got Event [Event{name:TIMEOUT_EVENT, key:null, object:null}] from Queue
2016-11-02 23:51:15,520 DEBUG [PeerFSMImpl] (FSM-SPeer{Uri=aaa://10.225.20.206; State=null; con=null; incConnull }_8-7) Process event [Event{name:TIMEOUT_EVENT, key:null, object:null}]. Peer State is [SUSPECT]
2016-11-02 23:51:15,520 DEBUG [SCTPServerConnection] (FSM-SPeer{Uri=aaa://10.225.20.206; State=null; con=null; incConnull }_8-7) Disconnecting SCTP Server Connection aaa://10.225.20.206:16018