File tree Expand file tree Collapse file tree 2 files changed +24
-6
lines changed Expand file tree Collapse file tree 2 files changed +24
-6
lines changed Original file line number Diff line number Diff line change
1
+ When using filters with DataFrames or Datasets, the
2
+ underlying MongoDB Connector code constructs an :manual: `aggregation
3
+ pipeline </core/aggregation-pipeline/> ` to filter the data in
4
+ MongoDB before sending it to Spark. This improves Spark performance
5
+ by retrieving and processing only the data you need.
6
+
7
+ MongoDB Spark Connector turns the following filters into
8
+ aggregation pipeline stages:
9
+
10
+ - And
11
+ - EqualNullSafe
12
+ - EqualTo
13
+ - GreaterThan
14
+ - GreaterThanOrEqual
15
+ - In
16
+ - IsNull
17
+ - LessThan
18
+ - LessThanOrEqual
19
+ - Not
20
+ - Or
21
+ - StringContains
22
+ - StringEndsWith
23
+ - StringStartsWith
Original file line number Diff line number Diff line change 1
1
Filters
2
2
-------
3
3
4
- .. note::
5
-
6
- When using filters with DataFrames or the Python API, the
7
- underlying Mongo Connector code constructs an :manual:`aggregation
8
- pipeline </core/aggregation-pipeline/>` to filter the data in
9
- MongoDB before sending it to Spark.
4
+ .. include:: includes/pushed-filters.rst
10
5
11
6
Use ``filter()`` to read a subset of data from your MongoDB collection.
12
7
You can’t perform that action at this time.
0 commit comments