Skip to content

Commit 8385101

Browse files
ianf-mongodbianf-mongodb
authored
DOCSP-30628 Migrate Sync Jobs Drawer (#11)
* DOCSP-30628 Migrate Sync Jobs Drawer
1 parent c68bb43 commit 8385101

File tree

6 files changed

+412
-0
lines changed

6 files changed

+412
-0
lines changed

‎source/code-generation/code-generation.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
.. _rm-code-generation:
2+
13
===============
24
Code Generation
35
===============

‎source/jobs/creating-jobs.txt

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,150 @@
33
=================
44
Create a Sync Job
55
=================
6+
7+
.. contents:: On this page
8+
:local:
9+
:backlinks: none
10+
:depth: 1
11+
:class: singlecol
12+
13+
Sync jobs are the worker processes responsible for transferring your
14+
data and schemas from a relational database to MongoDB. Create a new
15+
sync job from the :guilabel:`Data Migration` tab.
16+
17+
About this Task
18+
---------------
19+
20+
The URI credentials you provide when creating a sync job do not
21+
need to be the same as the credentials used when creating your project.
22+
23+
Before you Begin
24+
----------------
25+
26+
- :ref:`Create one or more mapping rules <rm-create-mapping-rules>` in your
27+
Relational Migrator project.
28+
- Prepare a :ref:`MongoDB URI <rm-mongodb-database-connection-strings>` and
29+
credentials that have read/write permissions on your destination database.
30+
31+
Steps
32+
-----
33+
34+
#. On the :guilabel:`Data Migration` tab, click :guilabel:`Create Sync Job`.
35+
Relational Migrator only runs one sync job at a time. If a sync job is in progress, this button is disabled.
36+
37+
#. On the :guilabel:`Connect Source DB` form, enter the connection details
38+
to create the JDBC URI for your relational database.
39+
40+
2a. Select a database type from the :guilabel:`Database type` drop-down.
41+
42+
2b. Enter a host IP or DNS name in the :guilabel:`Host` text box.
43+
44+
2c. Enter a port number in the :guilabel:`Port` text box.
45+
46+
2d. Enter a database name in the :guilabel:`Database` text field.
47+
48+
Depending on your relational database, this behavior varies:
49+
50+
.. list-table::
51+
:header-rows: 1
52+
53+
* - Database Type
54+
- Behavior
55+
* - Oracle
56+
- You must enter a database name and a :guilabel:`Service ID` or :guilabel:`SID`.
57+
* - SQL Server
58+
- Enter a database name, or leave blank to load all databases.
59+
* - MySQL
60+
- Enter a database name, or leave blank to load all databases.
61+
* - Postgres
62+
- Leaving database name blank loads schemas from the default database.
63+
64+
2e. Enter a username in the :guilabel:`Username` text box.
65+
66+
2f. Enter a password in the :guilabel:`Password` text box.
67+
68+
2g. (Optional) Click the :guilabel:`Save a password for this session` checkbox.
69+
70+
2h. Click the :guilabel:`SSL` toggle switch to enable or disable SSL and select an SSL mode.
71+
72+
2i. Click :guilabel:`Connect`.
73+
74+
.. note::
75+
76+
Connection and SSL details depend on the database type you are connecting to.
77+
In addition to the generic connection properties listed above, you may also need
78+
to select a :guilabel:`Identifier` for Oracle and :guilabel:`Authentication`
79+
for SQL Server.
80+
81+
If you want to specify the JDBC URI manually, click the
82+
:guilabel:`Enter URI manually` toggle switch on the :guilabel:`Connect SourceDB`
83+
form. For details, see :ref:`rm-relational-database-connection-strings`
84+
.
85+
86+
#. Click :guilabel:`Connect`.
87+
88+
#. On the :guilabel:`Connect Destination DB` form, input your
89+
:guilabel:`MongoDB connection string (URI)`.
90+
91+
If you leave any of the form fields for :guilabel:`Database`, :guilabel:`Username`,
92+
or :guilabel:`Password` blank, the values specified in the URI are used.
93+
94+
#. Click :guilabel:`Connect`.
95+
96+
#. On the :guilabel:`Migration Options` form, select your
97+
:guilabel:`Migration Options`:
98+
99+
.. list-table::
100+
:header-rows: 1
101+
102+
* - Migration Option
103+
- Description
104+
* - Mode
105+
- Defines the type of sync job.
106+
* - Drop destination collections before migration
107+
- Boolean. Indicates whether Relational Migrator drops a
108+
destination collection before transferring data.
109+
* - Stop after errors
110+
- Integer. Indicates the number of errors after which Relational
111+
Migrator stops the sync job.
112+
* - Verify migrated data
113+
- Boolean. If true, the sync engine checks the migrated data
114+
against the source database. Only supported for snapshot mode.
115+
116+
Once you have established the database connection and specified the
117+
job type on the :guilabel:`Migration Options` form, Relational Migrator
118+
conducts various checks to ensure that the database is configured
119+
correctly. If any configuration is missing, a banner appears indicating
120+
that the database is not properly configured, with a
121+
:guilabel:`GENERATE SCRIPT` button to download a SQL script.
122+
123+
This script includes the required configuration statements
124+
and additional instructions in the form of comments.
125+
126+
.. warning::
127+
128+
Before proceeding with starting a sync job:
129+
130+
1. Download the script.
131+
2. Carefully review its contents.
132+
3. Execute the statements.
133+
4. Follow any commented manual steps.
134+
135+
#. Click :guilabel:`Start`.
136+
137+
Next Steps
138+
----------
139+
140+
- :ref:`rm-monitor-jobs`
141+
- :ref:`rm-data-Verification`
142+
143+
Learn More
144+
----------
145+
146+
For detailed information regarding the configuration requirements for each database,
147+
see the follow Debezium reference links, which are internally utilized by Relational Migrator:
148+
149+
* `MySQL <https://debezium.io/documentation/reference/stable/connectors/mysql.html#setting-up-mysql>`__
150+
* `Oracle <https://debezium.io/documentation/reference/stable/connectors/oracle.html#_preparing_the_database>`__
151+
* `PostgreSQL <https://debezium.io/documentation/reference/stable/connectors/postgresql.html#setting-up-postgresql>`__
152+
* `SQL Server <https://debezium.io/documentation/reference/stable/connectors/sqlserver.html#setting-up-sqlserver>`__

‎source/jobs/monitoring-jobs.txt

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,161 @@
1+
.. _rm-monitor-jobs:
2+
13
==================
24
Monitor a Sync Job
35
==================
6+
7+
.. contents:: On this page
8+
:local:
9+
:backlinks: none
10+
:depth: 1
11+
:class: singlecol
12+
13+
You can monitor a sync job from the :guilabel:`History` pane on the
14+
:guilabel:`Data Migration` tab. The :guilabel:`History` pane contains
15+
information about in progress, completed, and failed sync jobs.
16+
17+
About this Task
18+
---------------
19+
20+
Sync jobs execute in a number of distinct stages, which are visible on
21+
the :guilabel:`Job overview` pane.
22+
23+
For example:
24+
25+
- A snapshot sync job includes a snapshot stage and, optionally, a verification stage.
26+
- A continuous sync job includes a snapshot stage, followed by a CDC stage.
27+
28+
Job Overview
29+
~~~~~~~~~~~~
30+
31+
High-level sync job information displays in the top section of the :guilabel:`Job overview` pane.
32+
33+
.. list-table::
34+
:header-rows: 1
35+
:widths: 30 70
36+
37+
* - Metric
38+
- Description
39+
40+
* - Source URI
41+
- The JDBC connection string used to connect to the source relational database.
42+
43+
* - Destination URI
44+
- The MongoDB connection string used to connect to the destination MongoDB database.
45+
46+
* - Job Mode
47+
- Indicates if the sync job is snapshot or continuous.
48+
49+
* - Status
50+
- Indicates the current or end status of the sync job.
51+
52+
Snapshot Stage
53+
~~~~~~~~~~~~~~
54+
55+
Snapshot stage information displays in the :guilabel:`Snapshot stage` section of the :guilabel:`Job overview` pane.
56+
57+
.. list-table::
58+
:header-rows: 1
59+
:widths: 30 70
60+
61+
* - Metric
62+
- Description
63+
64+
* - Snapshot stage status
65+
- Indicates the status of the snapshot stage.
66+
67+
* - Started
68+
- Date and time that the snapshot stage started.
69+
70+
* - Duration
71+
- Time taken for the snapshot stage to complete.
72+
73+
* - Tables migrated
74+
- Total number of tables migrated from the source database.
75+
76+
* - Rows migrated
77+
- Total number of rows migrated from all tables.
78+
79+
CDC Stage
80+
~~~~~~~~~
81+
82+
The CDC stage information is displayed in the :guilabel:`CDC stage` section
83+
of the :guilabel:`Job overview` pane. The CDC stage begins after the initial
84+
snapshot stage has completed and remains in the :guilabel:`RUNNING` status
85+
until it is explicitly completed, or the sync job encounters an unrecoverable error.
86+
87+
.. list-table::
88+
:header-rows: 1
89+
:widths: 30 70
90+
91+
* - Metric
92+
- Description
93+
94+
* - CDC stage status
95+
- Indicates the status of the CDC stage. When a CDC stage is actively replicating
96+
data changes, this status is :guilabel:`RUNNING`.
97+
98+
* - Started
99+
- Date and time that Relational Migrator started listening for data change events.
100+
101+
* - Last event time
102+
- Latest date and time that Relational Migrator detected a data change event.
103+
104+
* - Total events seen
105+
- Number of data change events Relational Migrator has detected since the CDC stage started.
106+
107+
* - Events in the last hour
108+
- Number of data change events Relational Migrator has detected in the last hour.
109+
110+
Issues
111+
~~~~~~
112+
113+
If your sync job has errors during execution, you can find the details
114+
in the :guilabel:`Issues` pane on the :guilabel:`Data Migration` screen.
115+
116+
.. list-table::
117+
:header-rows: 1
118+
:widths: 30 70
119+
120+
* - Metric
121+
- Description
122+
123+
* - Date & Time
124+
- The date and time that the issue occurred.
125+
126+
* - Table / Collection
127+
- The table or collection that caused the error. This is blank if the error is global.
128+
129+
* - Count
130+
- Total number of errors encountered.
131+
132+
* - Error detail
133+
- Detailed error message from the target deployment.
134+
135+
Steps
136+
-----
137+
138+
1. Click on a previous sync job in the :guilabel:`History` pane to load the :guilabel:`Overview` and :guilabel:`Issue` information.
139+
140+
:guilabel:`Snapshot stage` and :guilabel:`CDC stage` logging information display next to each respective section of the :guilabel:`Job overview` pane.
141+
142+
.. image:: /img/jobs/jobs-monitoring-job-history.png
143+
:alt: View Job History
144+
145+
|
146+
147+
2. Click the :guilabel:`>` icon under the :guilabel:`Issues` section to view sync job errors.
148+
149+
3. Review the debugging information in the :guilabel:`Error detail` column.
150+
151+
Next Steps
152+
----------
153+
154+
- :ref:`rm-code-generation`
155+
- :ref:`rm-add-calculated-fields`
156+
157+
Learn More
158+
----------
159+
160+
- :ref:`rm-stop-jobs`
161+
- :ref:`rm-data-Verification`

‎source/jobs/stopping-jobs.txt

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,64 @@
1+
.. _rm-stop-jobs:
2+
13
===============
24
Stop a Sync Job
35
===============
46

7+
.. contents:: On this page
8+
:local:
9+
:backlinks: none
10+
:depth: 1
11+
:class: singlecol
12+
13+
You can stop a sync job from the :guilabel:`Job Overview` pane on the
14+
:guilabel:`Data Migration` tab.
15+
16+
About this Task
17+
---------------
18+
19+
The effects of stopping a sync job on your destination database depend on
20+
the type and stage of the sync job:
21+
22+
- If you stop a snapshot sync job during the snapshot stage, any
23+
partially migrated data remains in the MongoDB database.
24+
- If you stop a continuous sync job during the CDC stage, the process
25+
ends gracefully, and Relational Migrator cleans up any temporary data.
26+
27+
Once initiated, the process of canceling or completing a sync job may take up to one minute.
28+
29+
Steps
30+
-----
31+
32+
To stop your sync job, follow the instructions below based on the current stage of execution:
33+
34+
.. tabs::
35+
36+
tabs:
37+
38+
- id: stop-snapshot
39+
name: Stop a Sync Job in Snapshot stage
40+
content: |
41+
42+
#. Open the :guilabel:`Data Migration` tab.
43+
#. Click on the sync job in the :guilabel:`History` pane.
44+
#. Click the red :guilabel:`Terminate` button in the upper-right corner of the screen.
45+
#. Click the red :guilabel:`Terminate` button on the pop-up modal.
46+
Once a sync job cancels, an :guilabel:`x` icon displays
47+
in the :guilabel:`History` pane, and the status of the sync
48+
job updates to :guilabel:`Snapshot cancelled`.
49+
50+
- id: stop-cdc
51+
name: Stop a Sync Job in CDC stage
52+
content: |
53+
#. Open the :guilabel:`Data Migration` tab.
54+
#. Click on the sync job in the :guilabel:`History` pane.
55+
#. Click the :guilabel:`Complete CDC` button in the upper-right corner of the screen.
56+
#. Click the green :guilabel:`Complete CDC` button on the pop-up modal.
57+
Once the sync job completes, a checkmark icon displays in the :guilabel:`History` pane, all stages display a status of :guilabel:`COMPLETED`, and the status of the sync job updates to :guilabel:`CDC completed`.
58+
59+
Learn More
60+
----------
61+
62+
- :ref:`rm-create-jobs`
63+
- :ref:`rm-monitor-jobs`
564

0 commit comments

Comments
 (0)