Hi,
this is the first time I try to create a windows cluster. I first created two server-2012 VM (machine names are chh-win2012-10.xxx.com IP 10.172.2.148 and chh-win2012-11.xxx.com IP 10.172.2.190) in server 2008's Hyper-v, added them to the domain, then:
1. installed failover cluster feature, and set both VM firewall's domain/private/public probile's inbound an outbound connections' to Allow.
2. create a failover cluster (DbCluster1) in the failover cluster manager with the first VM chh-win2012-10.xxx.com only, succeeded.
3. added the second VM chh-win2012-11.xxx.com to DbCluster1, failed with message in UI: the operation is taking longer than expected, elpsed time is 00:02:xx
4. run powershell 'get-clusterlog', in windows\cluster\reports\cluster.log, I can find message:
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] got event: Remote endpoint 10.172.2.190:~3343~ unreachable from 10.172.2.148:~3343~
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] Marking Route from 10.172.2.148:~3343~ to 10.172.2.190:~3343~ as down
5. more log are pasted as below, any help/ suggetion is appreciated :
=================
Node: | chh-win2012-11.xxx.com | |
Started | 3/8/2013 6:52:53 AM | |
Completed | 3/8/2013 6:59:17 AM |
cluster.
chh-win2012-11.
chh-win2012-11.
chh-win2012-11.xxxx.com has started.
functional member of the cluster.
The server 'chh-win2012-11.xxx.com' could
not be added to the cluster.
An error occurred while adding node
'chh-win2012-11.xxxx.com' to cluster 'DbCluster1'.
This operation returned because the timeout period expired
==============
00000ca8.000004d4::2013/03/08-14:54:01.056 INFO [RES] Network Name <Cluster Name>: Netbios: Slow Operation, FinishWithReply: 0
00000ca8.000004d4::2013/03/08-14:54:01.056 INFO [RES] Network Name: [NN] got sync reply: 0
00000ca8.000004d4::2013/03/08-14:54:01.056 INFO [RES] Network Name <Cluster Name>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00000c40.00000c4c::2013/03/08-14:54:03.483 INFO [ACCEPT] 0.0.0.0:~3343~: Accepted inbound connection from remote endpoint 10.172.2.190:~50236~.
00000c40.000010d0::2013/03/08-14:54:03.483 INFO [SV] New real route: local (10.172.2.148:~3343~) to remote (10.172.2.190:~50236~).
00000c40.000010d0::2013/03/08-14:54:03.483 INFO [SV] Got a new incoming stream from 10.172.2.190:~50236~
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [SV] Authentication and authorization were successful
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [SV] Security Handshake successful while obtaining SecurityContext for NetFT driver
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [VER] Got new TCP connection. Exchanging version data.
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [VER] Checking version compatibility for node chh-win2012-11 id 2 with following versions: highest [Major 7 Minor 9200 Upgrade 2 ClusterVersion 0x000723F0], lowest [Major 7 Minor 9200 Upgrade 2 ClusterVersion
0x000723F0].
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [VER] Version check passed: node and cluster highest supported versions match.
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [SV] Negotiating message security level.
00000c40.000010d0::2013/03/08-14:54:03.514 INFO [SV] Already protecting connection with message security level 'Sign'.
00000c40.000010d0::2013/03/08-14:54:03.514 INFO [FTI] Got new raw TCP/IP connection.
00000c40.000010d0::2013/03/08-14:54:03.514 INFO [FTI][Initiator] This node (1) is initiator
00000c40.000010d0::2013/03/08-14:54:03.521 INFO [CHANNEL 10.172.2.190:~50236~] graceful close, status (of previous failure, may not indicate problem) ERROR_SUCCESS(0)
00000c40.000010d0::2013/03/08-14:54:03.583 INFO [CORE] Node 1: Clearing cookie 549204c5-63eb-4b58-80d1-469f2d2e64fc
00000c40.000010d0::2013/03/08-14:54:03.661 WARN mscs::ListenerWorker::operator (): GracefulClose(1226)' because of 'channel to remote endpoint 10.172.2.190:~50236~ is closed'
00000c40.00001338::2013/03/08-14:54:05.637 INFO [IM] got event: LocalEndpoint 10.172.2.148:~3343~ has missed two consecutive heartbeats from 10.172.2.190:~3343~
00000c40.00001338::2013/03/08-14:54:05.637 INFO [CHM] Received notification for two consecutive missed HBs to the remote endpoint 10.172.2.190:~3343~ from 10.172.2.148:~3343~
00000ca8.00000e8c::2013/03/08-14:54:06.057 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:58e0ea71-bf6b-422f-b1b3-7a0cb28a21ba:Netbios
00000ca8.000004d4::2013/03/08-14:54:06.057 INFO [RES] Network Name <Cluster Name>: Netbios: Slow Operation, FinishWithReply: 0
00000ca8.000004d4::2013/03/08-14:54:06.057 INFO [RES] Network Name: [NN] got sync reply: 0
00000ca8.000004d4::2013/03/08-14:54:06.057 INFO [RES] Network Name <Cluster Name>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00000c40.00000480::2013/03/08-14:54:08.640 DBG [NETFTAPI] Signaled NetftRemoteUnreachable event, local address 10.172.2.148:3343 remote address 10.172.2.190:3343
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] got event: Remote endpoint 10.172.2.190:~3343~ unreachable from 10.172.2.148:~3343~
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] Marking Route from 10.172.2.148:~3343~ to 10.172.2.190:~3343~ as down
00000c40.00001338::2013/03/08-14:54:08.640 INFO [NDP] Checking to see if all routes for route (virtual) local fe80::e126:4028:77b2:5f3f:~0~ to remote fe80::f87f:2067:9cc1:b59c:~0~ are down
00000c40.00001338::2013/03/08-14:54:08.640 INFO [NDP] All routes for route (virtual) local fe80::e126:4028:77b2:5f3f:~0~ to remote fe80::f87f:2067:9cc1:b59c:~0~ are down
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] Route history 1: Old: 05.976, Message: Request, Route sequence: 61, Received sequence: 61, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2013/03/08-06:54:02.663, Ticks since
last sending: 1
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] Route history 2: Old: 05.993, Message: Response, Route sequence: 61, Received sequence: 61, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2013/03/08-06:54:02.647, Ticks since
last sending: 0
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] Route history 3: Old: 05.993, Message: Request, Route sequence: 61, Received sequence: 61, Heartbeats counter/threshold: 4/5, Error: Success, NtStatus: 0 Timestamp: 2013/03/08-06:54:02.647, Ticks since
last sending: 0
00000ca8.000004d4::2013/03/08-14:54:01.056 INFO [RES] Network Name <Cluster Name>: Netbios: Slow Operation, FinishWithReply: 0
00000ca8.000004d4::2013/03/08-14:54:01.056 INFO [RES] Network Name: [NN] got sync reply: 0
00000ca8.000004d4::2013/03/08-14:54:01.056 INFO [RES] Network Name <Cluster Name>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00000c40.00000c4c::2013/03/08-14:54:03.483 INFO [ACCEPT] 0.0.0.0:~3343~: Accepted inbound connection from remote endpoint 10.172.2.190:~50236~.
00000c40.000010d0::2013/03/08-14:54:03.483 INFO [SV] New real route: local (10.172.2.148:~3343~) to remote (10.172.2.190:~50236~).
00000c40.000010d0::2013/03/08-14:54:03.483 INFO [SV] Got a new incoming stream from 10.172.2.190:~50236~
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [SV] Authentication and authorization were successful
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [SV] Security Handshake successful while obtaining SecurityContext for NetFT driver
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [VER] Got new TCP connection. Exchanging version data.
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [VER] Checking version compatibility for node chh-win2012-11 id 2 with following versions: highest [Major 7 Minor 9200 Upgrade 2 ClusterVersion 0x000723F0], lowest [Major 7 Minor 9200 Upgrade 2 ClusterVersion
0x000723F0].
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [VER] Version check passed: node and cluster highest supported versions match.
00000c40.000010d0::2013/03/08-14:54:03.499 INFO [SV] Negotiating message security level.
00000c40.000010d0::2013/03/08-14:54:03.514 INFO [SV] Already protecting connection with message security level 'Sign'.
00000c40.000010d0::2013/03/08-14:54:03.514 INFO [FTI] Got new raw TCP/IP connection.
00000c40.000010d0::2013/03/08-14:54:03.514 INFO [FTI][Initiator] This node (1) is initiator
00000c40.000010d0::2013/03/08-14:54:03.521 INFO [CHANNEL 10.172.2.190:~50236~] graceful close, status (of previous failure, may not indicate problem) ERROR_SUCCESS(0)
00000c40.000010d0::2013/03/08-14:54:03.583 INFO [CORE] Node 1: Clearing cookie 549204c5-63eb-4b58-80d1-469f2d2e64fc
00000c40.000010d0::2013/03/08-14:54:03.661 WARN mscs::ListenerWorker::operator (): GracefulClose(1226)' because of 'channel to remote endpoint 10.172.2.190:~50236~ is closed'
00000c40.00001338::2013/03/08-14:54:05.637 INFO [IM] got event: LocalEndpoint 10.172.2.148:~3343~ has missed two consecutive heartbeats from 10.172.2.190:~3343~
00000c40.00001338::2013/03/08-14:54:05.637 INFO [CHM] Received notification for two consecutive missed HBs to the remote endpoint 10.172.2.190:~3343~ from 10.172.2.148:~3343~
00000ca8.00000e8c::2013/03/08-14:54:06.057 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:58e0ea71-bf6b-422f-b1b3-7a0cb28a21ba:Netbios
00000ca8.000004d4::2013/03/08-14:54:06.057 INFO [RES] Network Name <Cluster Name>: Netbios: Slow Operation, FinishWithReply: 0
00000ca8.000004d4::2013/03/08-14:54:06.057 INFO [RES] Network Name: [NN] got sync reply: 0
00000ca8.000004d4::2013/03/08-14:54:06.057 INFO [RES] Network Name <Cluster Name>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00000c40.00000480::2013/03/08-14:54:08.640 DBG [NETFTAPI] Signaled NetftRemoteUnreachable event, local address 10.172.2.148:3343 remote address 10.172.2.190:3343
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] got event: Remote endpoint 10.172.2.190:~3343~ unreachable from 10.172.2.148:~3343~
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] Marking Route from 10.172.2.148:~3343~ to 10.172.2.190:~3343~ as down
00000c40.00001338::2013/03/08-14:54:08.640 INFO [NDP] Checking to see if all routes for route (virtual) local fe80::e126:4028:77b2:5f3f:~0~ to remote fe80::f87f:2067:9cc1:b59c:~0~ are down
00000c40.00001338::2013/03/08-14:54:08.640 INFO [NDP] All routes for route (virtual) local fe80::e126:4028:77b2:5f3f:~0~ to remote fe80::f87f:2067:9cc1:b59c:~0~ are down
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] Route history 1: Old: 05.976, Message: Request, Route sequence: 61, Received sequence: 61, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2013/03/08-06:54:02.663, Ticks since
last sending: 1
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] Route history 2: Old: 05.993, Message: Response, Route sequence: 61, Received sequence: 61, Heartbeats counter/threshold: 5/5, Error: Success, NtStatus: 0 Timestamp: 2013/03/08-06:54:02.647, Ticks since
last sending: 0
00000c40.00001338::2013/03/08-14:54:08.640 INFO [IM] Route history 3: Old: 05.993, Message: Request, Route sequence: 61, Received sequence: 61, Heartbeats counter/threshold: 4/5, Error: Success, NtStatus: 0 Timestamp: 2013/03/08-06:54:02.647, Ticks since
last sending: 0
app