Hi,
I have a Windows 2008 2 node cluster.
We have SQL on node 1. The cluster group is on node 2 right now.
Over the weekend, the cluster group failed and will not come back online. We were unable to connect via cluster manager through the name. Able to connect on the server, but only see cluster events and cluster status 'down'. The cluster service
on this node is on. Here is cluster GROUP output:
Cluster IP Address - Offline
Cluster Name - Offline
Quorum - Failed
When attempting to online the group we receive:
System error 5908 has occurred (0x00001714).
The group is unable to accept the request since it is moving to another node.
The cluster log contains many instances of this error:
00000b30.00001b7c::2014/08/18-16:31:07.502 ERR [RCM] rcm::RcmResControl::DoResourceControl: ERROR_RESOURCE_CALL_TIMED_OUT(5910)' because of 'Failed to wait for pending resource control call to Quorum.
The Q: drive is active on node 2 and I am able to browse and write files to it.
Here is the last 10 minutes of cluster log. I tried to online the group during that time. Thanks for your review.
-------------------------------------------------------------------------------------------------------------
00000b30.00002728::2014/08/18-16:24:59.737 ERR [RCM] rcm::RcmResControl::DoResourceControl: ERROR_RESOURCE_CALL_TIMED_OUT(5910)' because of 'Failed to wait for pending resource control call to Quorum.
'
00000b30.00002728::2014/08/18-16:24:59.737 WARN [RCM] ResourceControl(GET_COMMON_PROPERTIES) to Quorum returned 5910.
00000b30.0000138c::2014/08/18-16:25:17.883 ERR [RCM] rcm::RcmResControl::DoResourceControl: ERROR_RESOURCE_CALL_TIMED_OUT(5910)' because of 'Failed to wait for pending resource control call to Quorum.
'
00000b30.0000138c::2014/08/18-16:25:17.883 WARN [RCM] ResourceControl(STORAGE_GET_DISK_INFO_EX) to Quorum returned 5910.
000012c4.000018c0::2014/08/18-16:26:07.689 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqclus.dll
00000b30.000022f8::2014/08/18-16:26:07.689 INFO [RCM] rcm::RcmResType::LoadDll: Got error 126; will attempt to load mqclus.dll via Wow64.
00001330.00001968::2014/08/18-16:26:07.689 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqclus.dll
00000b30.000022f8::2014/08/18-16:26:07.689 WARN [RCM] Failed to load restype MSMQ: error 126.
000012c4.000018c0::2014/08/18-16:26:07.689 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqtgclus.dll
00000b30.000022f8::2014/08/18-16:26:07.689 INFO [RCM] rcm::RcmResType::LoadDll: Got error 126; will attempt to load mqtgclus.dll via Wow64.
00001330.00001968::2014/08/18-16:26:07.689 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqtgclus.dll
00000b30.000022f8::2014/08/18-16:26:07.689 WARN [RCM] Failed to load restype MSMQTriggers: error 126.
00000b30.00000f74::2014/08/18-16:26:25.305 ERR [RCM] rcm::RcmResControl::DoResourceControl: ERROR_RESOURCE_CALL_TIMED_OUT(5910)' because of 'Failed to wait for pending resource control call to Quorum.
'
00000b30.00000f74::2014/08/18-16:26:25.305 WARN [RCM] ResourceControl(GET_CLASS_INFO) to Quorum returned 5910.
00000b30.000006dc::2014/08/18-16:26:41.844 INFO [RCM] rcm::RcmApi::OnlineGroup: (Cluster Group)
00000b30.000006dc::2014/08/18-16:26:41.844 INFO [GUM] Node 2: Processing RequestLock 2:207
00000b30.00002618::2014/08/18-16:26:41.922 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285421)
00000b30.000006dc::2014/08/18-16:26:41.922 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:26:42.031 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:26:43.046 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:26:43.046 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:26:44.060 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:26:44.060 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:26:45.074 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:26:45.074 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:26:46.088 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:26:46.088 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:26:47.102 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:26:47.102 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:26:48.117 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:26:48.132 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:26:49.146 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:26:49.146 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:26:50.161 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:26:50.176 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:26:51.190 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:26:51.206 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00002618::2014/08/18-16:26:54.295 INFO [GUM] Node 2: Processing RequestLock 1:22512
00000b30.00002618::2014/08/18-16:26:54.295 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285431)
00000b30.000006dc::2014/08/18-16:27:01.223 INFO [GUM] Node 2: Processing RequestLock 2:217
00000b30.00002618::2014/08/18-16:27:01.223 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285432)
00000b30.000006dc::2014/08/18-16:27:01.223 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:27:01.223 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:27:11.241 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:27:11.241 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:27:21.258 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:27:21.289 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00002618::2014/08/18-16:27:23.614 INFO [GUM] Node 2: Processing RequestLock 1:22513
00000b30.00002618::2014/08/18-16:27:23.614 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285435)
00000b30.000006dc::2014/08/18-16:27:31.306 INFO [GUM] Node 2: Processing RequestLock 2:220
00000b30.00002618::2014/08/18-16:27:31.306 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285436)
00000b30.000006dc::2014/08/18-16:27:31.306 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:27:31.306 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:27:41.323 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:27:41.355 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:27:51.372 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:27:51.372 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00002618::2014/08/18-16:27:54.305 INFO [GUM] Node 2: Processing RequestLock 1:22514
00000b30.00002618::2014/08/18-16:27:54.305 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285439)
00000b30.000006dc::2014/08/18-16:28:01.389 INFO [GUM] Node 2: Processing RequestLock 2:223
00000b30.00002618::2014/08/18-16:28:01.389 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285440)
00000b30.000006dc::2014/08/18-16:28:01.389 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:28:01.389 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:28:11.406 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:28:11.422 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:28:21.439 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:28:21.455 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00002618::2014/08/18-16:28:23.624 INFO [GUM] Node 2: Processing RequestLock 1:22515
00000b30.00002618::2014/08/18-16:28:23.624 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285443)
00000b30.000006dc::2014/08/18-16:28:31.472 INFO [GUM] Node 2: Processing RequestLock 2:226
00000b30.00002618::2014/08/18-16:28:31.472 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285444)
00000b30.000006dc::2014/08/18-16:28:31.472 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:28:31.488 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:28:41.504 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:28:41.504 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:28:51.520 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:28:51.520 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00002618::2014/08/18-16:28:54.328 INFO [GUM] Node 2: Processing RequestLock 1:22516
00000b30.00002618::2014/08/18-16:28:54.328 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285447)
00000b30.000006dc::2014/08/18-16:29:01.536 INFO [GUM] Node 2: Processing RequestLock 2:229
00000b30.00002618::2014/08/18-16:29:01.536 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285448)
00000b30.000006dc::2014/08/18-16:29:01.536 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:29:01.552 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:29:11.568 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:29:11.568 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:29:21.584 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:29:21.600 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00002618::2014/08/18-16:29:23.643 INFO [GUM] Node 2: Processing RequestLock 1:22517
00000b30.00002618::2014/08/18-16:29:23.643 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285451)
00000b30.000006dc::2014/08/18-16:29:31.616 INFO [GUM] Node 2: Processing RequestLock 2:232
00000b30.00002618::2014/08/18-16:29:31.616 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285452)
00000b30.000006dc::2014/08/18-16:29:31.616 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:29:31.616 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:29:41.632 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:29:41.632 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:29:51.648 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:29:51.695 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00002618::2014/08/18-16:29:54.331 INFO [GUM] Node 2: Processing RequestLock 1:22518
00000b30.00002618::2014/08/18-16:29:54.331 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285455)
00000b30.000006dc::2014/08/18-16:30:01.711 INFO [GUM] Node 2: Processing RequestLock 2:235
00000b30.00002618::2014/08/18-16:30:01.711 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285456)
00000b30.000006dc::2014/08/18-16:30:01.711 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:30:01.711 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:30:11.727 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:30:11.742 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.0000138c::2014/08/18-16:30:17.889 ERR [RCM] rcm::RcmResControl::DoResourceControl: ERROR_RESOURCE_CALL_TIMED_OUT(5910)' because of 'Failed to wait for pending resource control call to Quorum.
'
00000b30.0000138c::2014/08/18-16:30:17.889 WARN [RCM] ResourceControl(GET_COMMON_PROPERTIES) to Quorum returned 5910.
00000b30.00001fdc::2014/08/18-16:30:17.905 INFO [NM] Received request from client address 10.12.13.8.
00000b30.00000f74::2014/08/18-16:30:17.905 INFO [NM] Received request from client address 10.12.13.8.
00000b30.0000115c::2014/08/18-16:30:17.952 INFO [NM] Received request from client address 10.12.13.8.
000012c4.000018c0::2014/08/18-16:30:18.326 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqclus.dll
00000b30.00000f74::2014/08/18-16:30:18.326 INFO [RCM] rcm::RcmResType::LoadDll: Got error 126; will attempt to load mqclus.dll via Wow64.
00001330.00000a10::2014/08/18-16:30:18.326 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqclus.dll
00000b30.00000f74::2014/08/18-16:30:18.326 WARN [RCM] Failed to load restype MSMQ: error 126.
00000b30.00000f74::2014/08/18-16:30:18.326 WARN [RCM] rcm::RcmApi::ResTypeControl: ResType MSMQ's DLL is not present on this node. Attempting to find a good node...
000012c4.000018c0::2014/08/18-16:30:18.326 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqclus.dll
00000b30.00002120::2014/08/18-16:30:18.326 INFO [RCM] rcm::RcmResType::LoadDll: Got error 126; will attempt to load mqclus.dll via Wow64.
00001330.00000a10::2014/08/18-16:30:18.326 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqclus.dll
00000b30.00002120::2014/08/18-16:30:18.326 WARN [RCM] Failed to load restype MSMQ: error 126.
000012c4.000018c0::2014/08/18-16:30:18.326 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqtgclus.dll
00000b30.00000f74::2014/08/18-16:30:18.326 INFO [RCM] rcm::RcmResType::LoadDll: Got error 126; will attempt to load mqtgclus.dll via Wow64.
00001330.00000a10::2014/08/18-16:30:18.326 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqtgclus.dll
00000b30.00000f74::2014/08/18-16:30:18.326 WARN [RCM] Failed to load restype MSMQTriggers: error 126.
00000b30.00000f74::2014/08/18-16:30:18.326 WARN [RCM] rcm::RcmApi::ResTypeControl: ResType MSMQTriggers's DLL is not present on this node. Attempting to find a good node...
000012c4.000018c0::2014/08/18-16:30:18.326 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqtgclus.dll
00000b30.00002120::2014/08/18-16:30:18.326 INFO [RCM] rcm::RcmResType::LoadDll: Got error 126; will attempt to load mqtgclus.dll via Wow64.
00001330.00000a10::2014/08/18-16:30:18.326 WARN [RHS] ERROR_MOD_NOT_FOUND(126), unable to load resource DLL mqtgclus.dll
00000b30.00002120::2014/08/18-16:30:18.326 WARN [RCM] Failed to load restype MSMQTriggers: error 126.
00000b30.000006dc::2014/08/18-16:30:21.759 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:30:21.759 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00002618::2014/08/18-16:30:23.646 INFO [GUM] Node 2: Processing RequestLock 1:22519
00000b30.00002618::2014/08/18-16:30:23.646 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285459)
00000b30.000006dc::2014/08/18-16:30:31.775 INFO [GUM] Node 2: Processing RequestLock 2:238
00000b30.00002618::2014/08/18-16:30:31.775 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285460)
00000b30.000006dc::2014/08/18-16:30:31.775 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:30:31.775 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000021dc::2014/08/18-16:30:33.974 INFO [NM] Received request from client address 10.12.13.8.
00000b30.000006dc::2014/08/18-16:30:41.791 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:30:41.822 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:30:51.838 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:30:51.838 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00002618::2014/08/18-16:30:54.334 INFO [GUM] Node 2: Processing RequestLock 1:22520
00000b30.00002618::2014/08/18-16:30:54.334 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285463)
00000b30.000006dc::2014/08/18-16:31:01.854 INFO [GUM] Node 2: Processing RequestLock 2:241
00000b30.00002618::2014/08/18-16:31:01.854 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285464)
00000b30.000006dc::2014/08/18-16:31:01.854 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:31:01.870 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00001b7c::2014/08/18-16:31:07.502 ERR [RCM] rcm::RcmResControl::DoResourceControl: ERROR_RESOURCE_CALL_TIMED_OUT(5910)' because of 'Failed to wait for pending resource control call to Quorum.
'
00000b30.00001b7c::2014/08/18-16:31:07.502 WARN [RCM] ResourceControl(STORAGE_GET_DISK_INFO_EX) to Quorum returned 5910.
00000b30.000019f0::2014/08/18-16:31:07.861 ERR [RCM] rcm::RcmResControl::DoResourceControl: ERROR_RESOURCE_CALL_TIMED_OUT(5910)' because of 'Failed to wait for pending resource control call to Quorum.
'
00000b30.000019f0::2014/08/18-16:31:07.861 WARN [RCM] ResourceControl(GET_COMMON_PROPERTIES) to Quorum returned 5910.
00000b30.000006dc::2014/08/18-16:31:11.886 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:31:11.886 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000006dc::2014/08/18-16:31:21.902 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:31:21.902 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.00002618::2014/08/18-16:31:23.665 INFO [GUM] Node 2: Processing RequestLock 1:22521
00000b30.00002618::2014/08/18-16:31:23.665 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285467)
00000b30.000006dc::2014/08/18-16:31:31.918 INFO [GUM] Node 2: Processing RequestLock 2:244
00000b30.00002618::2014/08/18-16:31:31.918 INFO [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 285468)
00000b30.000006dc::2014/08/18-16:31:31.918 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(Cluster Group,1)
00000b30.000006dc::2014/08/18-16:31:31.949 WARN [RCM] rcm::RcmApi::OnlineGroup: retrying: Cluster Group, 5908.
00000b30.000022f8::2014/08/18-16:31:46.427 ERR [RCM] rcm::RcmResControl::DoResourceControl: ERROR_RESOURCE_CALL_TIMED_OUT(5910)' because of 'Failed to wait for pending resource control call to Quorum.
'
00000b30.000022f8::2014/08/18-16:31:46.427 WARN [RCM] ResourceControl(STORAGE_GET_DISK_INFO_EX) to Quorum returned 5910.
00000b30.00002618::2014/08/18-16:31:54.337 INFO [GUM] Node 2: Processing RequestLock 1:22522
00000b30.00002618::2014/08/18-16:31:54.337 INFO [GUM] Node 2: Processing GrantLock to 1 (sent by 2 gumid: 285469)