@@ -289,9 +289,6 @@ You must specify this partitioner using the full classname:
289
289
290
290
This partitioner creates a single partition.
291
291
292
-
293
-
294
-
295
292
.. _spark-change-stream-conf:
296
293
297
294
Change Streams
@@ -329,6 +326,20 @@ Change Streams
329
326
see the MongoDB server manual guide
330
327
:manual:`Lookup Full Document for Update Operation </changeStreams/#lookup-full-document-for-update-operations>`.
331
328
329
+ * - ``change.stream.micro.batch.max.partition.count``
330
+ - | The maximum number of partitions the {+connector-short+} divides each
331
+ micro-batch into. Spark workers can process these partitions in parallel.
332
+ |
333
+ | This setting applies only when using micro-batch streams.
334
+ |
335
+ | **Default**: ``1``
336
+
337
+ .. warning:: Event Order
338
+
339
+ Specifying a value larger than ``1`` can alter the order in which
340
+ the {+connector-short+} processes change events. Avoid this setting
341
+ if out-of-order processing could create data inconsistencies downstream.
342
+
332
343
* - ``change.stream.publish.full.document.only``
333
344
- | Specifies whether to publish the changed document or the full
334
345
change stream document.
@@ -348,20 +359,29 @@ Change Streams
348
359
This setting overrides the ``change.stream.lookup.full.document``
349
360
setting.
350
361
351
- * - ``change.stream.micro.batch.max.partition.count``
352
- - | The maximum number of partitions the {+connector-short+} divides each
353
- micro-batch into. Spark workers can process these partitions in parallel.
354
- |
355
- | This setting applies only when using micro-batch streams.
356
- |
357
- | **Default**: ``1``
358
-
359
- .. warning:: Event Order
360
-
361
- Specifying a value larger than ``1`` can alter the order in which
362
- the {+connector-short+} processes change events. Avoid this setting
363
- if out-of-order processing could create data inconsistencies downstream.
364
-
362
+ * - ``change.stream.startup.mode``
363
+ - | Specifies how the connector starts up when no offset is available.
364
+
365
+ | This setting accepts the following values:
366
+
367
+ - ``latest``: The connector begins processing
368
+ change events starting with the most recent event.
369
+ It will not process any earlier unprocessed events.
370
+ - ``timestamp``: The connector begins processing change events at a specified time.
371
+
372
+ To use the ``timestamp`` option, you must specify a time by using the
373
+ ``change.stream.startup.mode.timestamp.start.at.operation.time`` setting.
374
+ This setting accepts timestamps in the following formats:
375
+
376
+ - An integer representing the number of seconds since the
377
+ :wikipedia:`Unix epoch <Unix_time>`
378
+ - A date and time in
379
+ `ISO-8601 <https://www.iso.org/iso-8601-date-and-time-format.html>`__
380
+ format with one-second precision
381
+ - An extended JSON ``BsonTimestamp``
382
+
383
+ **Default**: ``latest``
384
+
365
385
.. _configure-input-uri:
366
386
367
387
``connection.uri`` Configuration Setting
0 commit comments