Merge Chunks in a Sharded Cluster

Overview

Changed in version 3.0.

The mergeChunks command allows you to combine continuouschunks on the same shard into a single chunk. This tutorial explainshow to merge neighboring chunks in a sharded cluster.

Procedure

Note

Examples in this procedure use a members collection in thetest database, using the username field as theshard key.

Identify Chunk Ranges

In the mongo shell, identify the chunkranges with the following operation:

  1. sh.status()

In the output, the chunk ranges appear after the chunk counts for eachsharded collection, as in the following example:

  1. --- Sharding Status ---
  2.   sharding version: {
  3.      "_id" : 1,
  4.      "minCompatibleVersion" : 5,
  5.      "currentVersion" : 6,
  6.      "clusterId" : ObjectId("5c114e68325aee6f60669228")
  7.   }
  8.   shards:
  9.         {  "_id" : "shardA",  "host" : "shardA/shardA-m1.example.net:27018,shardA-m2.example.net:27018,shardA-m3.example.net:27018",  "state" : 1 }
  10.         {  "_id" : "shardB",  "host" : "shardB/shardB-m1.example.net:27018,shardB-m2.example.net:27018,shardB-m3.example.net:27018",  "state" : 1 }
  11.   active mongoses:
  12.         "4.0.4" : 1
  13.   autosplit:
  14.         Currently enabled: yes
  15.   balancer:
  16.         Currently enabled:  yes
  17.         Currently running:  no
  18.         Failed balancer rounds in last 5 attempts:  0
  19.         Migration Results for the last 24 hours:
  20.                 8 : Success
  21.   databases:
  22.         {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
  23.         {  "_id" : "test",  "primary" : "shardA",  "partitioned" : true,  "version" : {  "uuid" : UUID("2632036c-bd74-457f-a364-5def901fe5f6"),  "lastMod" : 1 } }
  24.              test.members
  25.                   shard key: { "username" : 1 }
  26.                   unique: false
  27.                   balancing: true
  28.                   chunks:
  29.                           shardA   7
  30.                           shardB   7
  31.                   { "username" : { "$minKey" : 1 } } -->> { "username" : "user16643" } on : shardB Timestamp(2, 0)
  32.                   { "username" : "user16643" } -->> { "username" : "user2329" } on : shardB Timestamp(3, 0)
  33.                   { "username" : "user2329" } -->> { "username" : "user29937" } on : shardB Timestamp(4, 0)
  34.                   { "username" : "user29937" } -->> { "username" : "user36583" } on : shardB Timestamp(5, 0)
  35.                   { "username" : "user36583" } -->> { "username" : "user43229" } on : shardB Timestamp(6, 0)
  36.                   { "username" : "user43229" } -->> { "username" : "user49877" } on : shardB Timestamp(7, 0)
  37.                   { "username" : "user49877" } -->> { "username" : "user56522" } on : shardB Timestamp(8, 0)
  38.                   { "username" : "user56522" } -->> { "username" : "user63169" } on : shardA Timestamp(8, 1)
  39.                   { "username" : "user63169" } -->> { "username" : "user69816" } on : shardA Timestamp(1, 17)
  40.                   { "username" : "user69816" } -->> { "username" : "user76462" } on : shardA Timestamp(1, 19)
  41.                   { "username" : "user76462" } -->> { "username" : "user83108" } on : shardA Timestamp(1, 21)
  42.                   { "username" : "user83108" } -->> { "username" : "user89756" } on : shardA Timestamp(1, 23)
  43.                   { "username" : "user89756" } -->> { "username" : "user96401" } on : shardA Timestamp(1, 25)
  44.                   { "username" : "user96401" } -->> { "username" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 26)

The chunk ranges appear after the chunk counts for each shardedcollection. For example, the following are the chunk ranges for thetest.members collection:

  1. { "username" : { "$minKey" : 1 } } -->> { "username" : "user16643" } on : shardB Timestamp(2, 0)
  2. { "username" : "user16643" } -->> { "username" : "user2329" } on : shardB Timestamp(3, 0)
  3. { "username" : "user2329" } -->> { "username" : "user29937" } on : shardB Timestamp(4, 0)
  4. { "username" : "user29937" } -->> { "username" : "user36583" } on : shardB Timestamp(5, 0)
  5. { "username" : "user36583" } -->> { "username" : "user43229" } on : shardB Timestamp(6, 0)
  6. { "username" : "user43229" } -->> { "username" : "user49877" } on : shardB Timestamp(7, 0)
  7. { "username" : "user49877" } -->> { "username" : "user56522" } on : shardB Timestamp(8, 0)
  8. { "username" : "user56522" } -->> { "username" : "user63169" } on : shardA Timestamp(8, 1)
  9. { "username" : "user63169" } -->> { "username" : "user69816" } on : shardA Timestamp(1, 17)
  10. { "username" : "user69816" } -->> { "username" : "user76462" } on : shardA Timestamp(1, 19)
  11. { "username" : "user76462" } -->> { "username" : "user83108" } on : shardA Timestamp(1, 21)
  12. { "username" : "user83108" } -->> { "username" : "user89756" } on : shardA Timestamp(1, 23)
  13. { "username" : "user89756" } -->> { "username" : "user96401" } on : shardA Timestamp(1, 25)
  14. { "username" : "user96401" } -->> { "username" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 26)

Merge Chunks

Merge contiguous chunks on the same shard.

For example, consider the following chunk ranges on shardA:

Note

The chunks to be merged are highlighted.

  1. { "username" : "user56522" } -->> { "username" : "user63169" } on : shardA Timestamp(8, 1)
  2. { "username" : "user63169" } -->> { "username" : "user69816" } on : shardA Timestamp(1, 17)
  3. { "username" : "user69816" } -->> { "username" : "user76462" } on : shardA Timestamp(1, 19)
  4. { "username" : "user76462" } -->> { "username" : "user83108" } on : shardA Timestamp(1, 21)
  5. { "username" : "user83108" } -->> { "username" : "user89756" } on : shardA Timestamp(1, 23)
  6. { "username" : "user89756" } -->> { "username" : "user96401" } on : shardA Timestamp(1, 25)
  7. { "username" : "user96401" } -->> { "username" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 26)

To merge the highlighted contiguous chunks, issue themergeChunks command against the admin database:

  1. db.adminCommand( {
  2.    mergeChunks: "test.members",
  3.    bounds: [ { "username" : "user69816" },
  4.              { "username" : "user96401" } ]
  5. } )

On success, mergeChunks produces the following output:

  1. {
  2.    "ok" : 1,
  3.    "operationTime" : Timestamp(1544636362, 12),
  4.    "$clusterTime" : {
  5.       "clusterTime" : Timestamp(1544636362, 13),
  6.       "signature" : {
  7.          "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
  8.          "keyId" : NumberLong(0)
  9.       }
  10.    }
  11. }

On any failure condition, mergeChunks returns a documentwhere the value of the ok field is 0.

View Merged Chunks Ranges

After merging the identified chunks, confirm the new chunk, as follows:

  1. sh.status()

The output of sh.status() should resemble:

  1. --- Sharding Status ---
  2.   sharding version: {
  3.      "_id" : 1,
  4.      "minCompatibleVersion" : 5,
  5.      "currentVersion" : 6,
  6.      "clusterId" : ObjectId("5c114e68325aee6f60669228")
  7.   }
  8.   shards:
  9.         {  "_id" : "shardA",  "host" : "shardA/shardA-m1.example.net:27018,shardA-m2.example.net:27018,shardA-m3.example.net:27018",  "state" : 1 }
  10.         {  "_id" : "shardB",  "host" : "shardB/shardB-m1.example.net:27018,shardB-m2.example.net:27018,shardB-m3.example.net:27018",  "state" : 1 }
  11.   active mongoses:
  12.         "4.0.4" : 1
  13.   autosplit:
  14.         Currently enabled: yes
  15.   balancer:
  16.         Currently enabled:  yes
  17.         Currently running:  yes
  18.         Collections with active migrations:
  19.                 test.members started at Wed Dec 12 2018 13:14:55 GMT-0500 (EST)
  20.         Failed balancer rounds in last 5 attempts:  0
  21.         Migration Results for the last 24 hours:
  22.                 8 : Success
  23.   databases:
  24.         {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
  25.              config.system.sessions
  26.                   shard key: { "_id" : 1 }
  27.                   unique: false
  28.                   balancing: true
  29.                   chunks:
  30.                           shardA   1
  31.                   { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 0)
  32.         {  "_id" : "test",  "primary" : "shardA",  "partitioned" : true,  "version" : {  "uuid" : UUID("2632036c-bd74-457f-a364-5def901fe5f6"),  "lastMod" : 1 } }
  33.              test.members
  34.                   shard key: { "username" : 1 }
  35.                   unique: false
  36.                   balancing: true
  37.                   chunks:
  38.                           shardA   4
  39.                           shardB   7
  40.                   { "username" : { "$minKey" : 1 } } -->> { "username" : "user16643" } on : shardB Timestamp(2, 0)
  41.                   { "username" : "user16643" } -->> { "username" : "user2329" } on : shardB Timestamp(3, 0)
  42.                   { "username" : "user2329" } -->> { "username" : "user29937" } on : shardB Timestamp(4, 0)
  43.                   { "username" : "user29937" } -->> { "username" : "user36583" } on : shardB Timestamp(5, 0)
  44.                   { "username" : "user36583" } -->> { "username" : "user43229" } on : shardB Timestamp(6, 0)
  45.                   { "username" : "user43229" } -->> { "username" : "user49877" } on : shardB Timestamp(7, 0)
  46.                   { "username" : "user49877" } -->> { "username" : "user56522" } on : shardB Timestamp(8, 0)
  47.                   { "username" : "user56522" } -->> { "username" : "user63169" } on : shardA Timestamp(8, 1)
  48.                   { "username" : "user63169" } -->> { "username" : "user69816" } on : shardA Timestamp(1, 17)
  49.                   { "username" : "user69816" } -->> { "username" : "user96401" } on : shardA Timestamp(8, 2)
  50.                   { "username" : "user96401" } -->> { "username" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 26)

After the merge, the balancer may migratechunks across shards to ensure a more even distribution of chunks.