1.

What is the process to setup sharded cluster?

Answer»

The balancer is a background process that runs on the primary of config server in a cluster. It constantly monitors the number of CHUNKS on each shard and if the number of chunks for a specific shard is more than the migration threshold, it tries to automatically migrate chunks between shards so that there are an equal number of chunks per shard. The balancer migrates chunks from shards having more chunks to shards with lesser chunks. For example, Suppose we have 2 shards[shard01, shard02] with chunks 4 and 5 respectively. Now suppose there is a need to add another shard[shard03]. Initially, shard03 will have no chunks. A balancer will notice this UNEVEN distribution and migrate chunks from shard01 and shard02 to shard03 until all 3 shards have three shards each.

There might be performance impact when balancer migrates the chunks as they CARRY some overhead in TERMS of bandwidth and workload, which can impact database performance. To minimize the impact balancer:

  1. Attempts only one chunk migration at a given time. So a shard cannot participate in multiple chunk migrations at a given time. Multiple chink migrations should occur one after the other. Although for 3.4 parallel chunk migrations are possible. Suppose a sharded cluster has 4 shards it can participate in 2[shard/2] simultaneous chunk migrations.
  2. Kicks off balancing round only when a number of chunks between shard with the greatest and lowest number of shards reaches migration threshold.

Impact of Adding and Removing Shards on a balancer

Adding or removing the shard from the cluster creates imbalance as either new shard will have no chunks or removed shard chunks need to be redistributed throughout the cluster.  In case shard was removed from the cluster with uneven chunk distribution the balancer will remove the chunks from draining shard before balancing remaining uneven chunks. When balancer notices this imbalance it starts chunk migration process immediately. The migration process takes time to complete.



Discussion

No Comment Found