Skip to content

Resharding #5521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions config/redirects
Original file line number Diff line number Diff line change
Expand Up @@ -1811,6 +1811,13 @@ raw: /manual/core/wildcard -> ${base}/manual/core/index-wildcard/
[v5.0-*]: /${version}/tutorial/model-time-data -> ${base}/${version}/tutorial/model-iot-data/
[v3.6-v4.4]: /${version}/tutorial/model-iot-data -> ${base}/${version}/tutorial/model-time-data/

[v3.6-v4.4]: /${version}/core/sharding-shard-a-collection.txt -> ${base}/${version}/core/sharding-shard-key.txt
[v3.6-v4.4]: /${version}/core/sharding-choose-a-shard-key.txt -> ${base}/${version}/core/sharding-shard-key.txt
[v3.6-v4.4]: /${version}/core/sharding-refine-a-shard-key.txt -> ${base}/${version}/core/sharding-shard-key.txt
[v3.6-v4.4]: /${version}/core/sharding-change-shard-key-value.txt -> ${base}/${version}/core/sharding-shard-key.txt
[v3.6-v4.4]: /${version}/core/sharding-set-missing-shard-key-fields.txt -> ${base}/${version}/core/sharding-shard-key.txt
[v3.6-v4.4]: /${version}/core/sharding-find-shard-key.txt -> ${base}/${version}/core/sharding-shard-key.txt

#
# Upgrade / Downgrade redirects
#
Expand Down
2 changes: 2 additions & 0 deletions snooty.toml
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,10 @@ toc_landing_pages = [
"/core/security-users",
"/core/security-x.509",
"/core/sharded-cluster-components",
"/core/sharding-change-a-shard-key",
"/core/sharding-balancer-administration",
"/core/sharding-data-partitioning",
"/core/sharding-shard-key",
"/core/storage-engines",
"/core/timeseries-collections",
"/core/transactions",
Expand Down
15 changes: 7 additions & 8 deletions source/administration/production-checklist-development.txt
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,14 @@ structures. See :doc:`/core/data-models` for more information.
indexes required to support your queries. With the exception of
the ``_id`` index, you must create all indexes explicitly: MongoDB
does not automatically create any indexes other than ``_id``.

- Ensure that your schema design supports your deployment type: if
you planning to use :term:`sharded clusters <sharded cluster>` for
horizontal scaling, design your schema to include a strong shard
key. The shard key affects read and write performance by
determining how MongoDB partitions data. See: :doc:`Impacts of
Shard Keys on Cluster Operations </core/sharding-shard-key>`
for information about what qualities a shard key should possess.
You cannot change the shard key once it is set.
you are planning to use :term:`sharded clusters <sharded cluster>`
for horizontal scaling, design your schema to include a strong
shard key. While you can :ref:`change your shard key
<change-a-shard-key>` later, it is important to carefully consider
your :ref:`shard key choice <sharding-shard-key-requirements>` to
avoid scalability and perfomance issues.

- Ensure that your schema design does not rely on indexed arrays that
grow in length without bound. Typically, best performance can
Expand Down
5 changes: 3 additions & 2 deletions source/core/data-model-operations.txt
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,9 @@ To distribute data and application traffic in a sharded collection,
MongoDB uses the :ref:`shard key <shard-key>`. Selecting the proper
:ref:`shard key <shard-key>` has significant implications for
performance, and can enable or prevent query isolation and increased
write capacity. It is important to consider carefully the field or
fields to use as the shard key.
write capacity. While you can :ref:`change your shard key
<change-a-shard-key>` later, it is important to carefully consider your
shard key choice.

See :doc:`/sharding` and
:doc:`/core/sharding-shard-key` for more information.
Expand Down
2 changes: 2 additions & 0 deletions source/core/distributed-queries.txt
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ For more information on replica sets and write operations, see
:doc:`/reference/write-concern`.


.. _read-operations-sharded-clusters:

Read Operations to Sharded Clusters
-----------------------------------

Expand Down
25 changes: 11 additions & 14 deletions source/core/hashed-sharding.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Hashed Sharding
Hashed sharding uses either a :ref:`single field hashed index
<index-hashed-index>` or a :ref:`compound hashed index
<index-type-compound-hashed>` (*New in 4.4*) as the shard key to
partition data across your cluster.
partition data across your cluster.

Sharding on a Single Field Hashed Index
Hashed sharding provides a more even data distribution across the
Expand All @@ -35,7 +35,7 @@ Sharding on a Compound Hashed Index

Compound hashed index compute the hash value of a single field in the
compound index; this value is used along with the other fields in the
index as your shard key.
index as your shard key.

Compound hashed sharding supports features like :ref:`zone sharding
<zone-sharding>`, where the prefix (i.e. first) non-hashed field or
Expand Down Expand Up @@ -103,31 +103,28 @@ to use as the :term:`shard key`.

sh.shardCollection( "database.collection", { <field> : "hashed" } )

To shard a collection on a
To shard a collection on a
:ref:`compound hashed index <index-type-compound-hashed>`, specify
the full namespace of the collection and the target compound
hashed index to use as the :term:`shard key`:

.. code-block:: javascript

sh.shardCollection(
"database.collection",
"database.collection",
{ "fieldA" : 1, "fieldB" : 1, "fieldC" : "hashed" }
)

.. important::

- Starting in MongoDB 4.4, you can refine a collection's shard key
by adding a suffix field or fields to the existing key. In earlier
versions, once you shard a collection, the selection of the shard
key is immutable; i.e. you cannot select a different shard key for
that collection. For details on refining a shard key, see
:ref:`shard-key-refine`.
- Starting in MongoDB 5.0, you can :ref:`reshard a collection
<sharding-resharding>` by changing a collection's shard key.
- Starting in MongoDB 4.4, you can :ref:`refine a shard key
<shard-key-refine>` by adding a suffix field or fields to the
existing shard key.
- In MongoDB 4.2 and earlier, the choice of shard key cannot
be changed after sharding.

- .. include:: /includes/limits-sharding-shardkey-document-immutable.rst

For details on updating the shard key, see :ref:`update-shard-key`.

Shard a Populated Collection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
18 changes: 8 additions & 10 deletions source/core/ranged-sharding.txt
Original file line number Diff line number Diff line change
Expand Up @@ -52,16 +52,14 @@ to use as the :term:`shard key`.

.. important::

- Starting in MongoDB 4.4, you can refine a collection's shard key
by adding a suffix field or fields to the existing key. In earlier
versions, once you shard a collection, the selection of the shard
key is immutable; i.e. you cannot select a different shard key for
that collection. For details on refining a shard key, see
:ref:`shard-key-refine`.

- .. include:: /includes/limits-sharding-shardkey-document-immutable.rst

For details on updating the shard key, see :ref:`update-shard-key`.
- Starting in MongoDB 5.0, you can :ref:`reshard a collection
<sharding-resharding>` by changing a collection's shard key.
- Starting in MongoDB 4.4, you can :ref:`refine a shard key
<shard-key-refine>` by adding a suffix field or fields to the existing
shard key.
- In MongoDB 4.2 and earlier, the choice of shard key cannot
be changed after sharding.


Shard a Populated Collection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down
6 changes: 3 additions & 3 deletions source/core/sharded-cluster-config-servers.txt
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,6 @@ Replica Set Config Servers
.. include:: /includes/fact-config-server-replica-set-restrictions.rst


, config

.. _config-server-read-write-ops:

Read and Write Operations on Config Servers
Expand Down Expand Up @@ -143,7 +141,9 @@ Sharded Cluster Metadata

Config servers store metadata in the :doc:`/reference/config-database`.

.. important:: Always back up the ``config`` database before doing any
.. important::

Always back up the ``config`` database before doing any
maintenance on the config server.

To access the ``config`` database, issue the following command from the
Expand Down
43 changes: 43 additions & 0 deletions source/core/sharding-change-a-shard-key.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
.. _change-a-shard-key:

==================
Change a Shard Key
==================

.. default-domain:: mongodb

.. contents:: On this page
:local:
:backlinks: none
:depth: 2
:class: singlecol

The ideal shard key allows MongoDB to distribute documents evenly
throughout the cluster while facilitating common query patterns. A
suboptimal shard key can lead to uneven data distribution and the
following problems:

- :ref:`Jumbo chunks <sharding-troubleshooting-jumbo-chunks>`
- :ref:`Uneven load distribution
<sharding-troubleshooting-monotonicity>`
- :ref:`Decreased query performance over time
<sharding-troubleshooting-scatter-gather>`

To address these issues, MongoDB allows you to change your shard key:

- Starting in MongoDB 5.0, you can :ref:`reshard a collection
<sharding-resharding>` by changing a collection's shard key.
- Starting in MongoDB 4.4, you can :ref:`refine a shard key
<shard-key-refine>` by adding a suffix field or fields to the existing
shard key.

In MongoDB 4.2 and earlier, a document's shard key is immutable.

For more information on common performance and scaling issues and advice
on how to fix them, read :ref:`shardkey-troubleshoot-shard-keys`.

.. toctree::
:titlesonly:

/core/sharding-refine-a-shard-key.txt
/core/sharding-reshard-a-collection.txt
98 changes: 98 additions & 0 deletions source/core/sharding-change-shard-key-value.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
.. _update-shard-key:

===================================
Change a Document's Shard Key Value
===================================

.. default-domain:: mongodb

.. contents:: On this page
:local:
:backlinks: none
:depth: 2
:class: singlecol


Starting in MongoDB 4.2, you can update a document's shard key value
unless the shard key field is the immutable ``_id`` field.

.. important:: When updating the shard key value

- You **must** be on a :binary:`~bin.mongos`. Do **not** issue the
operation directly on the shard.

- You **must** run either in a :doc:`transaction
</core/transactions>` or as a :doc:`retryable write
</core/retryable-writes>`.

- You **must** include an equality condition on the full shard
key in the query filter. For example, consider a ``messages``
collection that uses ``{ activityid: 1, userid : 1 }`` as the
shard key. To update the shard key value for a document, you must
include ``activityid: <value>, userid: <value>`` in the query
filter. You can include additional fields in the query as
appropriate.

See also the specific write command/methods for additional
operation-specific requirements when run against a sharded
collection.

To update a shard key value, use the following operations:

.. list-table::
:header-rows: 1
:widths: 40 60

* - Command
- Method

* - :ref:`update <command-update-shard-key-modification>` with ``multi: false``
- | :ref:`db.collection.replaceOne() <replaceOne-shard-key-modification>`
| :ref:`db.collection.updateOne() <updateOne-shard-key-modification>`

To set to a non-``null`` value, the update :red:`must` be
performed either inside a transaction or as a retryable write.

* - :ref:`findAndModify <cmd-findAndModify-sharded-collection>`
- | :ref:`db.collection.findOneAndReplace() <findOneAndReplace-shard-key-modification>`
| :ref:`db.collection.findOneAndUpdate() <findOneAndUpdate-shard-key-modification>`
| :ref:`db.collection.findAndModify() <method-findAndModify-sharded-collection>`

To set to a non-``null`` value, the update :red:`must` be
performed either inside a transaction or as a retryable write.

* -
- | :method:`db.collection.bulkWrite()`
| :method:`Bulk.find.updateOne()`

If the shard key modification results in moving the document to
another shard, you cannot specify more than one shard key
modification in the bulk operation; the batch size has to be 1.

If the shard key modification does not result in moving the
document to another shard, you can specify multiple shard
key modification in the bulk operation.

To set to a non-``null`` value, the operation :red:`must` be
performed either inside a transaction or as a retryable write.

.. include:: /includes/shard-key-modification-warning.rst

Example
-------

Consider a ``sales`` collection which is sharded on the ``location``
field. The collection contains a document with the ``_id``
``12345`` and the ``location`` ``""``. To update the field value for
this document, you can run the following command:

.. code-block:: javascript

db.sales.updateOne(
{ _id: 12345, location: "" },
{ $set: { location: "New York"} }
)

.. seealso::

:ref:`shard-key-missing-set`
Loading