diff --git a/docs/client-concepts/client-concepts.asciidoc b/docs/client-concepts/client-concepts.asciidoc deleted file mode 100644 index 6b333b44615..00000000000 --- a/docs/client-concepts/client-concepts.asciidoc +++ /dev/null @@ -1,26 +0,0 @@ -[[client-concepts]] -= Client concepts - -The .NET client for {es} maps closely to the original {es} API. All -requests and responses are exposed through types, making it ideal for getting up and running quickly. - -[[serialization]] -== Serialization - -By default, the .NET client for {es} uses the Microsoft System.Text.Json library for serialization. The client understands how to serialize and -deserialize the request and response types correctly. It also handles (de)serialization of user POCO types representing documents read or written to {es}. - -The client has two distinct serialization responsibilities - serialization of the types owned by the `Elastic.Clients.Elasticsearch` library and serialization of source documents, modeled in application code. The first responsibility is entirely internal; the second is configurable. - -[[source-serialization]] -=== Source serialization - -Source serialization refers to the process of (de)serializing POCO types in consumer applications as source documents indexed and retrieved from {es}. A source serializer implementation handles serialization, with the default implementation using the `System.Text.Json` library. As a result, you may use `System.Text.Json` attributes and converters to control the serialization behavior. - -* <> - -* <> - -include::serialization/modeling-documents-with-types.asciidoc[] - -include::serialization/custom-serialization.asciidoc[] diff --git a/docs/client-concepts/serialization/custom-serialization.asciidoc b/docs/client-concepts/serialization/custom-serialization.asciidoc deleted file mode 100644 index 3ca00988e20..00000000000 --- a/docs/client-concepts/serialization/custom-serialization.asciidoc +++ /dev/null @@ -1,232 +0,0 @@ -[[customizing-source-serialization]] -==== Customizing source serialization - -The built-in source serializer handles most POCO document models correctly. Sometimes, you may need further control over how your types are serialized. - -NOTE: The built-in source serializer uses the https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/overview[Microsoft `System.Text.Json` library] internally. You can apply `System.Text.Json` attributes and converters to control the serialization of your document types. - -[discrete] -[[system-text-json-attributes]] -===== Using `System.Text.Json` attributes - -`System.Text.Json` includes attributes that can be applied to types and properties to control their serialization. These can be applied to your POCO document types to perform actions such as controlling the name of a property or ignoring a property entirely. Visit the https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/overview[Microsoft documentation for further examples]. - -We can model a document to represent data about a person using a regular class (POCO), applying `System.Text.Json` attributes as necessary. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=usings-serialization] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=person-class-with-attributes] ----- -<1> The `JsonPropertyName` attribute ensures the `FirstName` property uses the JSON name `forename` when serialized. -<2> The `JsonIgnore` attribute prevents the `Age` property from appearing in the serialized JSON. - -We can then index an instance of the document into {es}. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=usings] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=index-person-with-attributes] ----- - -The index request is serialized, with the source serializer handling the `Person` type, serializing the POCO property named `FirstName` to the JSON object member named `forename`. The `Age` property is ignored and does not appear in the JSON. - -[source,javascript] ----- -{ - "forename": "Steve" -} ----- - -[discrete] -[[configuring-custom-jsonserializeroptions]] -===== Configuring custom `JsonSerializerOptions` - -The default source serializer applies a set of standard `JsonSerializerOptions` when serializing source document types. In some circumstances, you may need to override some of our defaults. This is achievable by creating an instance of `DefaultSourceSerializer` and passing an `Action`, which is applied after our defaults have been set. This mechanism allows you to apply additional settings or change the value of our defaults. - -The `DefaultSourceSerializer` includes a constructor that accepts the current `IElasticsearchClientSettings` and a `configureOptions` `Action`. - -[source,csharp] ----- -public DefaultSourceSerializer(IElasticsearchClientSettings settings, Action configureOptions); ----- - -Our application defines the following `Person` class, which models a document we will index to {es}. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=person-class] ----- - -We want to serialize our source document using Pascal Casing for the JSON properties. Since the options applied in the `DefaultSouceSerializer` set the `PropertyNamingPolicy` to `JsonNamingPolicy.CamelCase`, we must override this setting. After configuring the `ElasticsearchClientSettings`, we index our document to {es}. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=usings] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=custom-options-local-function] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=create-client] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=index-person] ----- -<1> A local function can be defined, accepting a `JsonSerializerOptions` parameter. Here, we set `PropertyNamingPolicy` to `null`. This returns to the default behavior for `System.Text.Json`, which uses Pascal Case. -<2> When creating the `ElasticsearchClientSettings`, we supply a `SourceSerializerFactory` using a lambda. The factory function creates a new instance of `DefaultSourceSerializer`, passing in the `settings` and our `ConfigureOptions` local function. We have now configured the settings with a custom instance of the source serializer. - -The `Person` instance is serialized, with the source serializer serializing the POCO property named `FirstName` using Pascal Case. - -[source,javascript] ----- -{ - "FirstName": "Steve" -} ----- - -As an alternative to using a local function, we could store an `Action` into a variable instead, which can be passed to the `DefaultSouceSerializer` constructor. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=custom-options-action] ----- - -[discrete] -[[registering-custom-converters]] -===== Registering custom `System.Text.Json` converters - -In certain more advanced situations, you may have types which require further customization during serialization than is possible using `System.Text.Json` property attributes. In these cases, the recommendation from Microsoft is to leverage a custom `JsonConverter`. Source document types serialized using the `DefaultSourceSerializer` can leverage the power of custom converters. - -For this example, our application has a document class that should use a legacy JSON structure to continue operating with existing indexed documents. Several options are available, but we'll apply a custom converter in this case. - -Our class is defined, and the `JsonConverter` attribute is applied to the class type, specifying the type of a custom converter. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=usings-serialization] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=customer-with-jsonconverter-attribute] ----- -<1> The `JsonConverter` attribute signals to `System.Text.Json` that it should use a converter of type `CustomerConverter` when serializing instances of this class. - -When serializing this class, rather than include a string value representing the value of the `CustomerType` property, we must send a boolean property named `isStandard`. This requirement can be achieved with a custom JsonConverter implementation. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=converter-usings] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=customer-converter] ----- -<1> When reading, this converter reads the `isStandard` boolean and translate this to the correct `CustomerType` enum value. -<2> When writing, this converter translates the `CustomerType` enum value to an `isStandard` boolean property. - -We can then index a customer document into {es}. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=usings] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=index-customer-with-converter] ----- - -The `Customer` instance is serialized using the custom converter, creating the following JSON document. - -[source,javascript] ----- -{ - "customerName": "Customer Ltd", - "isStandard": false -} ----- - -[discrete] -[[creating-custom-system-text-json-serializer]] -===== Creating a custom `SystemTextJsonSerializer` - -The built-in `DefaultSourceSerializer` includes the registration of `JsonConverter` instances which apply during source serialization. In most cases, these provide the proper behavior for serializing source documents, including those which use `Elastic.Clients.Elasticsearch` types on their properties. - -An example of a situation where you may require more control over the converter registration order is for serializing `enum` types. The `DefaultSourceSerializer` registers the `System.Text.Json.Serialization.JsonStringEnumConverter`, so enum values are serialized using their string representation. Generally, this is the preferred option for types used to index documents to {es}. - -In some scenarios, you may need to control the string value sent for an enumeration value. That is not directly supported in `System.Text.Json` but can be achieved by creating a custom `JsonConverter` for the `enum` type you wish to customize. In this situation, it is not sufficient to use the `JsonConverterAttribute` on the `enum` type to register the converter. `System.Text.Json` will prefer the converters added to the `Converters` collection on the `JsonSerializerOptions` over an attribute applied to an `enum` type. It is, therefore, necessary to either remove the `JsonStringEnumConverter` from the `Converters` collection or register a specialized converter for your `enum` type before the `JsonStringEnumConverter`. - -The latter is possible via several techniques. When using the {es} .NET library, we can achieve this by deriving from the abstract `SystemTextJsonSerializer` class. - -Here we have a POCO which uses the `CustomerType` enum as the type for a property. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=usings-serialization] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=customer-without-jsonconverter-attribute] ----- - -To customize the strings used during serialization of the `CustomerType`, we define a custom `JsonConverter` specific to our `enum` type. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=usings-serialization] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=customer-type-converter] ----- -<1> When reading, this converter translates the string used in the JSON to the matching enum value. -<2> When writing, this converter translates the `CustomerType` enum value to a custom string value written to the JSON. - -We create a serializer derived from `SystemTextJsonSerializer` to give us complete control of converter registration order. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=derived-converter-usings] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=my-custom-serializer] ----- -<1> Inherit from `SystemTextJsonSerializer`. -<2> In the constructor, use the factory method `DefaultSourceSerializer.CreateDefaultJsonSerializerOptions` to create default options for serialization. No default converters are registered at this stage because we pass `false` as an argument. -<3> Register our `CustomerTypeConverter` as the first converter. -<4> To apply any default converters, call the `DefaultSourceSerializer.AddDefaultConverters` helper method, passing the options to modify. -<5> Implement the `CreateJsonSerializerOptions` method returning the stored `JsonSerializerOptions`. - -Because we have registered our `CustomerTypeConverter` before the default converters (which include the `JsonStringEnumConverter`), our converter takes precedence when serializing `CustomerType` instances on source documents. - -The base `SystemTextJsonSerializer` class handles the implementation details of binding, which is required to ensure that the built-in converters can access the `IElasticsearchClientSettings` where needed. - -We can then index a customer document into {es}. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=usings] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=index-customer-without-jsonconverter-attribute] ----- - -The `Customer` instance is serialized using the custom `enum` converter, creating the following JSON document. - -[source,javascript] ----- -{ - "customerName": "Customer Ltd", - "customerType": "premium" // <1> -} ----- -<1> The string value applied during serialization is provided by our custom converter. - -[discrete] -[[creating-custom-serializers]] -===== Creating a custom `Serializer` - -Suppose you prefer using an alternative JSON serialization library for your source types. In that case, you can inject an isolated serializer only to be called for the serialization of `_source`, `_fields`, or wherever a user-provided value is expected to be written and returned. - -Implementing `Elastic.Transport.Serializer` is technically enough to create a custom source serializer. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=vanilla-serializer-using-directives] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=vanilla-serializer] ----- - -Registering up the serializer is performed in the `ConnectionSettings` constructor. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=usings] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=register-vanilla-serializer] ----- -<1> If implementing `Serializer` is enough, why must we provide an instance wrapped in a factory `Func`? - -There are various cases where you might have a POCO type that contains an `Elastic.Clients.Elasticsearch` type as one of its properties. The `SourceSerializerFactory` delegate provides access to the default built-in serializer so you can access it when necessary. For example, consider if you want to use percolation; you need to store {es} queries as part of the `_source` of your document, which means you need to have a POCO that looks like this. - -[source,csharp] ----- -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=querydsl-using-directives] -include::{doc-tests-src}/ClientConcepts/Serialization/CustomSerializationTests.cs[tag=percolation-document] ----- - -A custom serializer would not know how to serialize `Query` or other `Elastic.Clients.Elasticsearch` types that could appear as part of -the `_source` of a document. Therefore, your custom `Serializer` would need to store a reference to our built-in serializer and delegate serialization of Elastic types back to it. \ No newline at end of file diff --git a/docs/client-concepts/serialization/modeling-documents-with-types.asciidoc b/docs/client-concepts/serialization/modeling-documents-with-types.asciidoc deleted file mode 100644 index d4e57b6d575..00000000000 --- a/docs/client-concepts/serialization/modeling-documents-with-types.asciidoc +++ /dev/null @@ -1,37 +0,0 @@ -[[modeling-documents-with-types]] -==== Modeling documents with types - -{es} provides search and aggregation capabilities on the documents that it is sent and indexes. These documents are sent as -JSON objects within the request body of a HTTP request. It is natural to model documents within the {es} .NET client using -https://en.wikipedia.org/wiki/Plain_Old_CLR_Object[POCOs (__Plain Old CLR Objects__)]. - -This section provides an overview of how types and type hierarchies can be used to model documents. - -[[default-behaviour]] -===== Default behaviour - -The default behaviour is to serialize type property names as camelcase JSON object members. - -We can model documents using a regular class (POCO). - -[source,csharp] ----- -include-tagged::{doc-tests-src}/ClientConcepts/Serialization/ModellingDocumentsWithTypesTests.cs[my-document-poco] ----- - -We can then index the an instance of the document into {es}. - -[source,csharp] ----- -include-tagged::{doc-tests-src}/ClientConcepts/Serialization/ModellingDocumentsWithTypesTests.cs[usings] -include-tagged::{doc-tests-src}/ClientConcepts/Serialization/ModellingDocumentsWithTypesTests.cs[index-my-document] ----- - -The index request is serialized, with the source serializer handling the `MyDocument` type, serializing the POCO property named `StringProperty` to the JSON object member named `stringProperty`. - -[source,javascript] ----- -{ - "stringProperty": "value" -} ----- \ No newline at end of file diff --git a/docs/client-concepts/troubleshooting/audit-trail.asciidoc b/docs/client-concepts/troubleshooting/audit-trail.asciidoc deleted file mode 100644 index 7622c09e41c..00000000000 --- a/docs/client-concepts/troubleshooting/audit-trail.asciidoc +++ /dev/null @@ -1,80 +0,0 @@ - - -:github: https://github.com/elastic/elasticsearch-net - -:nuget: https://www.nuget.org/packages - - -[[audit-trail]] -=== Audit trail - -Elasticsearch.Net and NEST provide an audit trail for the events within the request pipeline that -occur when a request is made. This audit trail is available on the response as demonstrated in the -following example. - -We'll use a Sniffing connection pool here since it sniffs on startup and pings before -first usage, so we can get an audit trail with a few events out - -[source,csharp] ----- -var pool = new SniffingConnectionPool(new []{ TestConnectionSettings.CreateUri() }); -var connectionSettings = new ConnectionSettings(pool) - .DefaultMappingFor(i => i - .IndexName("project") - ); - -var client = new ElasticClient(connectionSettings); ----- - -After issuing the following request - -[source,csharp] ----- -var response = client.Search(s => s - .MatchAll() -); ----- - -The audit trail is provided in the <> in a human -readable fashion, similar to - -.... -Valid NEST response built from a successful low level call on POST: /project/doc/_search -# Audit trail of this API call: - - [1] SniffOnStartup: Took: 00:00:00.0360264 - - [2] SniffSuccess: Node: http://localhost:9200/ Took: 00:00:00.0310228 - - [3] PingSuccess: Node: http://127.0.0.1:9200/ Took: 00:00:00.0115074 - - [4] HealthyResponse: Node: http://127.0.0.1:9200/ Took: 00:00:00.1477640 -# Request: - -# Response: - -.... - -to help with troubleshootin - -[source,csharp] ----- -var debug = response.DebugInformation; ----- - -But can also be accessed manually: - -[source,csharp] ----- -response.ApiCall.AuditTrail.Count.Should().Be(4, "{0}", debug); -response.ApiCall.AuditTrail[0].Event.Should().Be(SniffOnStartup, "{0}", debug); -response.ApiCall.AuditTrail[1].Event.Should().Be(SniffSuccess, "{0}", debug); -response.ApiCall.AuditTrail[2].Event.Should().Be(PingSuccess, "{0}", debug); -response.ApiCall.AuditTrail[3].Event.Should().Be(HealthyResponse, "{0}", debug); ----- - -Each audit has a started and ended `DateTime` on it that will provide -some understanding of how long it took - -[source,csharp] ----- -response.ApiCall.AuditTrail - .Should().OnlyContain(a => a.Ended - a.Started >= TimeSpan.Zero); ----- - diff --git a/docs/client-concepts/troubleshooting/capture-requests-localhost.png b/docs/client-concepts/troubleshooting/capture-requests-localhost.png deleted file mode 100644 index 3730681e5ed..00000000000 Binary files a/docs/client-concepts/troubleshooting/capture-requests-localhost.png and /dev/null differ diff --git a/docs/client-concepts/troubleshooting/capture-requests-remotehost.png b/docs/client-concepts/troubleshooting/capture-requests-remotehost.png deleted file mode 100644 index 26a913d2cbe..00000000000 Binary files a/docs/client-concepts/troubleshooting/capture-requests-remotehost.png and /dev/null differ diff --git a/docs/client-concepts/troubleshooting/debug-information.asciidoc b/docs/client-concepts/troubleshooting/debug-information.asciidoc deleted file mode 100644 index a7504312d2d..00000000000 --- a/docs/client-concepts/troubleshooting/debug-information.asciidoc +++ /dev/null @@ -1,180 +0,0 @@ - - -:github: https://github.com/elastic/elasticsearch-net - -:nuget: https://www.nuget.org/packages - -[[debug-information]] -=== Debug information - -Every response from Elasticsearch.Net and NEST contains a `DebugInformation` property -that provides a human readable description of what happened during the request for both successful and -failed requests - -[source,csharp] ----- -var response = client.Search(s => s - .Query(q => q - .MatchAll() - ) -); - -response.DebugInformation.Should().Contain("Valid NEST response"); ----- - -This can be useful in tracking down numerous problems and can also be useful when filing an -{github}/issues[issue] on the GitHub repository. - -==== Request and response bytes - -By default, the request and response bytes are not available within the debug information, but -can be enabled globally on Connection Settings by setting `DisableDirectStreaming`. This -disables direct streaming of - -. the serialized request type to the request stream - -. the response stream to a deserialized response type - -[source,csharp] ----- -var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); - -var settings = new ConnectionSettings(connectionPool) - .DisableDirectStreaming(); <1> - -var client = new ElasticClient(settings); ----- -<1> disable direct streaming for *all* requests - -or on a _per request_ basis - -[source,csharp] ----- -var response = client.Search(s => s - .RequestConfiguration(r => r - .DisableDirectStreaming() <1> - ) - .Query(q => q - .MatchAll() - ) -); ----- -<1> disable direct streaming for *this* request only - -Configuring `DisableDirectStreaming` on an individual request takes precedence over -any global configuration. - -There is typically a performance and allocation cost associated with disabling direct streaming -since both the request and response bytes must be buffered in memory, to allow them to be -exposed on the response call details. - -==== TCP statistics - -It can often be useful to see the statistics for active TCP connections, particularly when -trying to diagnose issues with the client. The client can collect the states of active TCP -connections just before making a request, and expose these on the response and in the debug -information. - -Similarly to `DisableDirectStreaming`, TCP statistics can be collected for every request -by configuring on `ConnectionSettings` - -[source,csharp] ----- -var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); - -var settings = new ConnectionSettings(connectionPool) - .EnableTcpStats(); <1> - -var client = new ElasticClient(settings); ----- -<1> collect TCP statistics for *all* requests - -or on a _per request_ basis - -[source,csharp] ----- -var response = client.Search(s => s - .RequestConfiguration(r => r - .EnableTcpStats() <1> - ) - .Query(q => q - .MatchAll() - ) -); - -var debugInformation = response.DebugInformation; ----- -<1> collect TCP statistics for *this* request only - -With `EnableTcpStats` set, the states of active TCP connections will now be included -on the response and in the debug information. - -The client includes a `TcpStats` -class to help with retrieving more detail about active TCP connections should it be -required - -[source,csharp] ----- -var tcpStatistics = TcpStats.GetActiveTcpConnections(); <1> -var ipv4Stats = TcpStats.GetTcpStatistics(NetworkInterfaceComponent.IPv4); <2> -var ipv6Stats = TcpStats.GetTcpStatistics(NetworkInterfaceComponent.IPv6); <3> - -var response = client.Search(s => s - .Query(q => q - .MatchAll() - ) -); ----- -<1> Retrieve details about active TCP connections, including local and remote addresses and ports -<2> Retrieve statistics about IPv4 -<3> Retrieve statistics about IPv6 - -[NOTE] --- -Collecting TCP statistics may not be accessible in all environments, for example, Azure App Services. -When this is the case, `TcpStats.GetActiveTcpConnections()` returns `null`. - --- - -==== ThreadPool statistics - -It can often be useful to see the statistics for thread pool threads, particularly when -trying to diagnose issues with the client. The client can collect statistics for both -worker threads and asynchronous I/O threads, and expose these on the response and -in debug information. - -Similar to collecting TCP statistics, ThreadPool statistics can be collected for all requests -by configuring `EnableThreadPoolStats` on `ConnectionSettings` - -[source,csharp] ----- -var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); - -var settings = new ConnectionSettings(connectionPool) - .EnableThreadPoolStats(); <1> - -var client = new ElasticClient(settings); ----- -<1> collect thread pool statistics for *all* requests - -or on a _per request_ basis - -[source,csharp] ----- -var response = client.Search(s => s - .RequestConfiguration(r => r - .EnableThreadPoolStats() <1> - ) - .Query(q => q - .MatchAll() - ) - ); - -var debugInformation = response.DebugInformation; <2> ----- -<1> collect thread pool statistics for *this* request only -<2> contains thread pool statistics - -With `EnableThreadPoolStats` set, the statistics of thread pool threads will now be included -on the response and in the debug information. - diff --git a/docs/client-concepts/troubleshooting/debug-mode.asciidoc b/docs/client-concepts/troubleshooting/debug-mode.asciidoc deleted file mode 100644 index 05d84092f3e..00000000000 --- a/docs/client-concepts/troubleshooting/debug-mode.asciidoc +++ /dev/null @@ -1,65 +0,0 @@ - - -:github: https://github.com/elastic/elasticsearch-net - -:nuget: https://www.nuget.org/packages - -[[debug-mode]] -=== Debug mode - -The <> explains that every response from Elasticsearch.Net -and NEST contains a `DebugInformation` property, and properties on `ConnectionSettings` and -`RequestConfiguration` can control which additional information is included in debug information, -for all requests or on a per request basis, respectively. - -During development, it can be useful to enable the most verbose debug information, to help -identify and troubleshoot problems, or simply ensure that the client is behaving as expected. -The `EnableDebugMode` setting on `ConnectionSettings` is a convenient shorthand for enabling -verbose debug information, configuring a number of settings like - -* disabling direct streaming to capture request and response bytes - -* prettyfying JSON responses from Elasticsearch - -* collecting TCP statistics when a request is made - -* collecting thread pool statistics when a request is made - -* including the Elasticsearch stack trace in the response if there is a an error on the server side - -[source,csharp] ----- -IConnectionPool pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); - -var settings = new ConnectionSettings(pool) - .EnableDebugMode(); <1> - -var client = new ElasticClient(settings); - -var response = client.Search(s => s - .Query(q => q - .MatchAll() - ) -); - -var debugInformation = response.DebugInformation; <2> ----- -<1> configure debug mode -<2> verbose debug information - -In addition to exposing debug information on the response, debug mode will also cause the debug -information to be written to the trace listeners in the `System.Diagnostics.Debug.Listeners` collection -by default, when the request has completed. A delegate can be passed when enabling debug mode to perform -a different action when a request has completed, using <> - -[source,csharp] ----- -var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); -var client = new ElasticClient(new ConnectionSettings(pool) - .EnableDebugMode(apiCallDetails => - { - // do something with the call details e.g. send with logging framework - }) -); ----- - diff --git a/docs/client-concepts/troubleshooting/deprecation-logging.asciidoc b/docs/client-concepts/troubleshooting/deprecation-logging.asciidoc deleted file mode 100644 index c69a0fc1ee1..00000000000 --- a/docs/client-concepts/troubleshooting/deprecation-logging.asciidoc +++ /dev/null @@ -1,40 +0,0 @@ - - -:github: https://github.com/elastic/elasticsearch-net - -:nuget: https://www.nuget.org/packages - -[[deprecation-logging]] -=== Deprecation logging - -Elasticsearch will send back `Warn` HTTP Headers when you are using an API feature that is -deprecated and will be removed or rewritten in a future release. - -Elasticsearch.NET and NEST report these back to you so you can log and watch out for them. - -[source,csharp] ----- -var request = new SearchRequest -{ - Size = 0, - Aggregations = new CompositeAggregation("test") - { - Sources = new [] - { - new DateHistogramCompositeAggregationSource("date") - { - Field = Field(f => f.LastActivity), - Interval = new Time("7d"), - Format = "yyyy-MM-dd" - } - } - } -}; -var response = this.Client.Search(request); - -response.ApiCall.DeprecationWarnings.Should().NotBeNullOrEmpty(); -response.ApiCall.DeprecationWarnings.Should().HaveCountGreaterOrEqualTo(1); -response.DebugInformation.Should().Contain("deprecated"); <1> ----- -<1> `DebugInformation` also contains the deprecation warnings - diff --git a/docs/client-concepts/troubleshooting/diagnostic-source.asciidoc b/docs/client-concepts/troubleshooting/diagnostic-source.asciidoc deleted file mode 100644 index 2f8e189cd60..00000000000 --- a/docs/client-concepts/troubleshooting/diagnostic-source.asciidoc +++ /dev/null @@ -1,121 +0,0 @@ - - -:github: https://github.com/elastic/elasticsearch-net - -:nuget: https://www.nuget.org/packages - -[[diagnostic-source]] -=== Diagnostic Source - -Elasticsearch.Net and NEST support capturing diagnostics information using `DiagnosticSource` and `Activity` from the -`System.Diagnostics` namespace. - -To aid with the discoverability of the topics you can subscribe to and the event names they emit, -both topics and event names are exposed as strongly typed strings under `Elasticsearch.Net.Diagnostics.DiagnosticSources` - -Subscribing to DiagnosticSources means implementing `IObserver` -or using `.Subscribe(observer, filter)` to opt in to the correct topic. - -Here we choose the more verbose `IObserver<>` implementation - -[source,csharp] ----- -public class ListenerObserver : IObserver, IDisposable -{ - private long _messagesWrittenToConsole = 0; - public long MessagesWrittenToConsole => _messagesWrittenToConsole; - - public Exception SeenException { get; private set; } - - public void OnError(Exception error) => SeenException = error; - public bool Completed { get; private set; } - public void OnCompleted() => Completed = true; - - private void WriteToConsole(string eventName, T data) - { - var a = Activity.Current; - Interlocked.Increment(ref _messagesWrittenToConsole); - } - - private List Disposables { get; } = new List(); - - public void OnNext(DiagnosticListener value) - { - void TrySubscribe(string sourceName, Func>> listener) <1> - { - if (value.Name != sourceName) return; - - var subscription = value.Subscribe(listener()); - Disposables.Add(subscription); - } - - TrySubscribe(DiagnosticSources.AuditTrailEvents.SourceName, - () => new AuditDiagnosticObserver(v => WriteToConsole(v.Key, v.Value))); - - TrySubscribe(DiagnosticSources.Serializer.SourceName, - () => new SerializerDiagnosticObserver(v => WriteToConsole(v.Key, v.Value))); - - TrySubscribe(DiagnosticSources.RequestPipeline.SourceName, - () => new RequestPipelineDiagnosticObserver( - v => WriteToConsole(v.Key, v.Value), - v => WriteToConsole(v.Key, v.Value) - )); - - TrySubscribe(DiagnosticSources.HttpConnection.SourceName, - () => new HttpConnectionDiagnosticObserver( - v => WriteToConsole(v.Key, v.Value), - v => WriteToConsole(v.Key, v.Value) - )); - } - - public void Dispose() - { - foreach(var d in Disposables) d.Dispose(); - } -} ----- -<1> By inspecting the name, we can selectively subscribe only to the topics `Elasticsearch.Net` emit - -Thanks to `DiagnosticSources`, you do not have to guess the topics emitted. - -The `DiagnosticListener.Subscribe` method expects an `IObserver>` -which is a rather generic message contract. As a subscriber, it's useful to know what `object` is in each case. -To help with this, each topic within the client has a dedicated `Observer` implementation that -takes an `onNext` delegate typed to the context object actually emitted. - -The RequestPipeline diagnostic source emits a different context objects the start and end of the `Activity` -For this reason, `RequestPipelineDiagnosticObserver` accepts two `onNext` delegates, -one for the `.Start` events and one for the `.Stop` events. - -[[subscribing-to-topics]] -==== Subscribing to topics - -As a concrete example of subscribing to topics, let's hook into all diagnostic sources and use -`ListenerObserver` to only listen to the ones from `Elasticsearch.Net` - -[source,csharp] ----- -using(var listenerObserver = new ListenerObserver()) -using (var subscription = DiagnosticListener.AllListeners.Subscribe(listenerObserver)) -{ - var pool = new SniffingConnectionPool(new []{ TestConnectionSettings.CreateUri() }); <1> - var connectionSettings = new ConnectionSettings(pool) - .DefaultMappingFor(i => i - .IndexName("project") - ); - - var client = new ElasticClient(connectionSettings); - - var response = client.Search(s => s <2> - .MatchAll() - ); - - listenerObserver.SeenException.Should().BeNull(); <3> - listenerObserver.Completed.Should().BeFalse(); - listenerObserver.MessagesWrittenToConsole.Should().BeGreaterThan(0); -} ----- -<1> use a sniffing connection pool that sniffs on startup and pings before first usage, so our diagnostics will emit most topics. -<2> make a search API call -<3> verify that the listener is picking up events - diff --git a/docs/client-concepts/troubleshooting/inspect-requests.png b/docs/client-concepts/troubleshooting/inspect-requests.png deleted file mode 100644 index 8b4a79b4093..00000000000 Binary files a/docs/client-concepts/troubleshooting/inspect-requests.png and /dev/null differ diff --git a/docs/client-concepts/troubleshooting/logging-with-fiddler.asciidoc b/docs/client-concepts/troubleshooting/logging-with-fiddler.asciidoc deleted file mode 100644 index 527f4bc53db..00000000000 --- a/docs/client-concepts/troubleshooting/logging-with-fiddler.asciidoc +++ /dev/null @@ -1,48 +0,0 @@ - - -:github: https://github.com/elastic/elasticsearch-net - -:nuget: https://www.nuget.org/packages - -[[logging-with-fiddler]] -=== Logging with Fiddler - -A web debugging proxy such as http://www.telerik.com/fiddler[Fiddler] is a useful way to capture HTTP traffic -from a machine, particularly whilst developing against a local Elasticsearch cluster. - -==== Capturing traffic to a remote cluster - -To capture traffic against a remote cluster is as simple as launching Fiddler! You may want to also -filter traffic to only show requests to the remote cluster by using the filters tab - -image::capture-requests-remotehost.png[Capturing requests to a remote host] - -==== Capturing traffic to a local cluster - -The .NET Framework is hardcoded not to send requests for `localhost` through any proxies and as a proxy -Fiddler will not receive such traffic. - -This is easily circumvented by using `ipv4.fiddler` as the hostname instead of `localhost` - -[source,csharp] ----- -var isFiddlerRunning = Process.GetProcessesByName("fiddler").Any(); -var host = isFiddlerRunning ? "ipv4.fiddler" : "localhost"; - -var connectionSettings = new ConnectionSettings(new Uri($"http://{host}:9200")) - .PrettyJson(); <1> - -var client = new ElasticClient(connectionSettings); ----- -<1> prettify json requests and responses to make them easier to read in Fiddler - -With Fiddler running, the requests and responses will now be captured and can be inspected in the -Inspectors tab - -image::inspect-requests.png[Inspecting requests and responses] - -As before, you may also want to filter traffic to only show requests to `ipv4.fiddler` on the port -on which you are running Elasticsearch. - -image::capture-requests-localhost.png[Capturing requests to localhost] - diff --git a/docs/client-concepts/troubleshooting/logging-with-on-request-completed.asciidoc b/docs/client-concepts/troubleshooting/logging-with-on-request-completed.asciidoc deleted file mode 100644 index a21c9ce67a0..00000000000 --- a/docs/client-concepts/troubleshooting/logging-with-on-request-completed.asciidoc +++ /dev/null @@ -1,219 +0,0 @@ - - -:github: https://github.com/elastic/elasticsearch-net - -:nuget: https://www.nuget.org/packages - -[[logging-with-on-request-completed]] -=== Logging with OnRequestCompleted - -When constructing the connection settings to pass to the client, you can pass a callback of type -`Action` to the `OnRequestCompleted` method that can eavesdrop every time a -response(good or bad) is received. - -If you have complex logging needs this is a good place to add that in -since you have access to both the request and response details. - -In this example, we'll use `OnRequestCompleted` on connection settings to increment a counter each time -it's called. - -[source,csharp] ----- -var counter = 0; -var client = new ElasticClient(new AlwaysInMemoryConnectionSettings().OnRequestCompleted(r => counter++)); <1> - -client.RootNodeInfo(); <2> -counter.Should().Be(1); - -await client.RootNodeInfoAsync(); <3> -counter.Should().Be(2); ----- -<1> Construct a client -<2> Make a synchronous call and assert the counter is incremented -<3> Make an asynchronous call and assert the counter is incremented - -`OnRequestCompleted` is called even when an exception is thrown, so it can be used even if the client is -configured to throw exceptions - -[source,csharp] ----- -var counter = 0; -var client = FixedResponseClient.Create( <1> - new { }, - 500, - connectionSettings => connectionSettings - .ThrowExceptions() <2> - .OnRequestCompleted(r => counter++) -); - -Assert.Throws(() => client.RootNodeInfo()); <3> -counter.Should().Be(1); - -await Assert.ThrowsAsync(async () => await client.RootNodeInfoAsync()); -counter.Should().Be(2); ----- -<1> Configure a client with a connection that **always returns a HTTP 500 response -<2> Always throw exceptions when a call results in an exception -<3> Assert an exception is thrown and the counter is incremented - -Here's an example using `OnRequestCompleted()` for more complex logging - -[NOTE] --- -By default, the client writes directly to the request stream and deserializes directly from the -response stream. - -If you would also like to capture the request and/or response bytes, -you also need to set `.DisableDirectStreaming()` to `true`. - --- - -[source,csharp] ----- -var list = new List(); -var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); - -var settings = new ConnectionSettings(connectionPool, new InMemoryConnection()) <1> - .DefaultIndex("default-index") - .DisableDirectStreaming() <2> - .OnRequestCompleted(apiCallDetails => <3> - { - // log out the request and the request body, if one exists for the type of request - if (apiCallDetails.RequestBodyInBytes != null) - { - list.Add( - $"{apiCallDetails.HttpMethod} {apiCallDetails.Uri} " + - $"{Encoding.UTF8.GetString(apiCallDetails.RequestBodyInBytes)}"); - } - else - { - list.Add($"{apiCallDetails.HttpMethod} {apiCallDetails.Uri}"); - } - - // log out the response and the response body, if one exists for the type of response - if (apiCallDetails.ResponseBodyInBytes != null) - { - list.Add($"Status: {apiCallDetails.HttpStatusCode}" + - $"{Encoding.UTF8.GetString(apiCallDetails.ResponseBodyInBytes)}"); - } - else - { - list.Add($"Status: {apiCallDetails.HttpStatusCode}"); - } - }); - -var client = new ElasticClient(settings); - -var syncResponse = client.Search(s => s <4> - .AllIndices() - .Scroll("2m") - .Sort(ss => ss - .Ascending(SortSpecialField.DocumentIndexOrder) - ) -); - -list.Count.Should().Be(2); - -var asyncResponse = await client.SearchAsync(s => s <5> - .AllIndices() - .Scroll("10m") - .Sort(ss => ss - .Ascending(SortSpecialField.DocumentIndexOrder) - ) -); - -list.Count.Should().Be(4); -list.Should().BeEquivalentTo(new[] <6> -{ - @"POST http://localhost:9200/_all/_search?typed_keys=true&scroll=2m {""sort"":[{""_doc"":{""order"":""asc""}}]}", - @"Status: 200", - @"POST http://localhost:9200/_all/_search?typed_keys=true&scroll=10m {""sort"":[{""_doc"":{""order"":""asc""}}]}", - @"Status: 200" -}); ----- -<1> Here we use `InMemoryConnection` but in a real application, you'd use an `IConnection` that _actually_ sends the request, such as `HttpConnection` -<2> Disable direct streaming so we can capture the request and response bytes -<3> Perform some action when a request completes. Here, we're just adding to a list, but in your application you may be logging to a file. -<4> Make a synchronous call -<5> Make an asynchronous call -<6> Assert the list contains the contents written in the delegate passed to `OnRequestCompleted` - -When running an application in production, you probably don't want to disable direct streaming for _all_ -requests, since doing so will incur a performance overhead, due to buffering request and -response bytes in memory. It can however be useful to capture requests and responses in an ad-hoc fashion, -perhaps to troubleshoot an issue in production. - -`DisableDirectStreaming` can be enabled on a _per-request_ basis for this purpose. In using this feature, -it is possible to configure a general logging mechanism in `OnRequestCompleted` and log out -request and responses only when necessary - -[source,csharp] ----- -var list = new List(); -var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200")); - -var settings = new ConnectionSettings(connectionPool, new InMemoryConnection()) - .DefaultIndex("default-index") - .OnRequestCompleted(apiCallDetails => - { - // log out the request and the request body, if one exists for the type of request - if (apiCallDetails.RequestBodyInBytes != null) - { - list.Add( - $"{apiCallDetails.HttpMethod} {apiCallDetails.Uri} " + - $"{Encoding.UTF8.GetString(apiCallDetails.RequestBodyInBytes)}"); - } - else - { - list.Add($"{apiCallDetails.HttpMethod} {apiCallDetails.Uri}"); - } - - // log out the response and the response body, if one exists for the type of response - if (apiCallDetails.ResponseBodyInBytes != null) - { - list.Add($"Status: {apiCallDetails.HttpStatusCode}" + - $"{Encoding.UTF8.GetString(apiCallDetails.ResponseBodyInBytes)}"); - } - else - { - list.Add($"Status: {apiCallDetails.HttpStatusCode}"); - } - }); - -var client = new ElasticClient(settings); - -var syncResponse = client.Search(s => s <1> - .AllIndices() - .Scroll("2m") - .Sort(ss => ss - .Ascending(SortSpecialField.DocumentIndexOrder) - ) -); - -list.Count.Should().Be(2); - -var asyncResponse = await client.SearchAsync(s => s <2> - .RequestConfiguration(r => r - .DisableDirectStreaming() - ) - .AllIndices() - .Scroll("10m") - .Sort(ss => ss - .Ascending(SortSpecialField.DocumentIndexOrder) - ) -); - -list.Count.Should().Be(4); -list.Should().BeEquivalentTo(new[] -{ - @"POST http://localhost:9200/_all/_search?typed_keys=true&scroll=2m", <3> - @"Status: 200", - @"POST http://localhost:9200/_all/_search?typed_keys=true&scroll=10m {""sort"":[{""_doc"":{""order"":""asc""}}]}", <4> - @"Status: 200" -}); ----- -<1> Make a synchronous call where the request and response bytes will not be buffered -<2> Make an asynchronous call where `DisableDirectStreaming()` is enabled -<3> Only the method and url for the first request is captured -<4> the body of the second request is captured - diff --git a/docs/configuration.asciidoc b/docs/configuration.asciidoc deleted file mode 100644 index a577b3e40b6..00000000000 --- a/docs/configuration.asciidoc +++ /dev/null @@ -1,247 +0,0 @@ -[[configuration]] -== Configuration - -Connecting to {es} with the client is easy, but it's possible that you'd like to -change the default connection behaviour. There are a number of configuration -options available on `ElasticsearchClientSettings` that can be used to control how the -client interact with {es}. - -=== Options on ElasticsearchClientSettings - -The following is a list of available connection configuration options on -`ElasticsearchClientSettings`: - -`Authentication`:: - -An implementation of `IAuthenticationHeader` describing what http header to use -to authenticate with the product. -+ - `BasicAuthentication` for basic authentication -+ - `ApiKey` for simple secret token -+ - `Base64ApiKey` for Elastic Cloud style encoded api keys - -`ClientCertificate`:: - -Use the following certificates to authenticate all HTTP requests. You can also -set them on individual request using `ClientCertificates`. - -`ClientCertificates`:: - -Use the following certificates to authenticate all HTTP requests. You can also -set them on individual request using `ClientCertificates`. - -`ConnectionLimit`:: - -Limits the number of concurrent connections that can be opened to an endpoint. -Defaults to 80 (see `DefaultConnectionLimit`). -+ -For Desktop CLR, this setting applies to the `DefaultConnectionLimit` property -on the `ServicePointManager` object when creating `ServicePoint` objects, -affecting the default `IConnection` implementation. -+ -For Core CLR, this setting applies to the `MaxConnectionsPerServer` property on -the `HttpClientHandler` instances used by the `HttpClient` inside the default -`IConnection` implementation. - -`DeadTimeout`:: - -The time to put dead nodes out of rotation (this will be multiplied by the -number of times they've been dead). - -`DefaultDisableIdInference`:: - -Disables automatic Id inference for given CLR types. -+ -The client by default will use the value of a property named `Id` on a CLR type -as the `_id` to send to {es}. Adding a type will disable this behaviour for that -CLR type. If `Id` inference should be disabled for all CLR types, use -`DefaultDisableIdInference`. - -`DefaultFieldNameInferrer`:: - -Specifies how field names are inferred from CLR property names. -+ -By default, the client camel cases property names. For example, CLR property -`EmailAddress` will be inferred as "emailAddress" {es} document field name. - -`DefaultIndex`:: - -The default index to use for a request when no index has been explicitly -specified and no default indices are specified for the given CLR type specified -for the request. - -`DefaultMappingFor`:: - -Specify how the mapping is inferred for a given CLR type. The mapping can infer -the index, id and relation name for a given CLR type, as well as control -serialization behaviour for CLR properties. - -`DisableAutomaticProxyDetection`:: - -Disabled proxy detection on the webrequest, in some cases this may speed up the -first connection your appdomain makes, in other cases it will actually increase -the time for the first connection. No silver bullet! Use with care! - -`DisableDirectStreaming`:: - -When set to true will disable (de)serializing directly to the request and -response stream and return a byte[] copy of the raw request and response. -Defaults to false. - -`DisablePing`:: - -This signals that we do not want to send initial pings to unknown/previously -dead nodes and just send the call straightaway. - -`DnsRefreshTimeout`:: - -DnsRefreshTimeout for the connections. Defaults to 5 minutes. - -`EnableDebugMode`:: - -Turns on settings that aid in debugging like `DisableDirectStreaming()` and -`PrettyJson()` so that the original request and response JSON can be inspected. -It also always asks the server for the full stack trace on errors. - -`EnableHttpCompression`:: - -Enable gzip compressed requests and responses. - -`EnableHttpPipelining`:: - -Whether HTTP pipelining is enabled. The default is `true`. - -`EnableTcpKeepAlive`:: - -Sets the keep-alive option on a TCP connection. -+ -For Desktop CLR, sets `ServicePointManager`.`SetTcpKeepAlive`. - -`EnableTcpStats`:: - -Enable statistics about TCP connections to be collected when making a request. - -`GlobalHeaders`:: - -Try to send these headers for every request. - -`GlobalQueryStringParameters`:: - -Append these query string parameters automatically to every request. - -`MaxDeadTimeout`:: - -The maximum amount of time a node is allowed to marked dead. - -`MaximumRetries`:: - -When a retryable exception occurs or status code is returned this controls the -maximum amount of times we should retry the call to {es}. - -`MaxRetryTimeout`:: - -Limits the total runtime including retries separately from `RequestTimeout`. -When not specified defaults to `RequestTimeout` which itself defaults to 60 -seconds. - -`MemoryStreamFactory`:: - -Provides a memory stream factory. - -`NodePredicate`:: - -Register a predicate to select which nodes that you want to execute API calls -on. Note that sniffing requests omit this predicate and always execute on all -nodes. When using an `IConnectionPool` implementation that supports reseeding of -nodes, this will default to omitting master only node from regular API calls. -When using static or single node connection pooling it is assumed the list of -node you instantiate the client with should be taken verbatim. - -`OnRequestCompleted`:: - -Allows you to register a callback every time a an API call is returned. - -`OnRequestDataCreated`:: - -An action to run when the `RequestData` for a request has been created. - -`PingTimeout`:: - -The timeout in milliseconds to use for ping requests, which are issued to -determine whether a node is alive. - -`PrettyJson`:: - -Provide hints to serializer and products to produce pretty, non minified json. -+ -Note: this is not a guarantee you will always get prettified json. - -`Proxy`:: - -If your connection has to go through proxy, use this method to specify the -proxy url. - -`RequestTimeout`:: - -The timeout in milliseconds for each request to {es}. - -`ServerCertificateValidationCallback`:: - -Register a `ServerCertificateValidationCallback` per request. - -`SkipDeserializationForStatusCodes`:: - -Configure the client to skip deserialization of certain status codes, for -example, you run {es} behind a proxy that returns an unexpected json format. - -`SniffLifeSpan`:: - -Force a new sniff for the cluster when the cluster state information is older -than the specified timespan. - -`SniffOnConnectionFault`:: - -Force a new sniff for the cluster state every time a connection dies. - -`SniffOnStartup`:: - -Sniff the cluster state immediately on startup. - -`ThrowExceptions`:: - -Instead of following a c/go like error checking on response. `IsValid` do throw -an exception (except when `SuccessOrKnownError` is false) on the client when a -call resulted in an exception on either the client or the {es} server. -+ -Reasons for such exceptions could be search parser errors, index missing -exceptions, and so on. - -`TransferEncodingChunked`:: - -Whether the request should be sent with chunked Transfer-Encoding. - -`UserAgent`:: - -The user agent string to send with requests. Useful for debugging purposes to -understand client and framework versions that initiate requests to {es}. - - -==== ElasticsearchClientSettings with ElasticsearchClient - -Here's an example to demonstrate setting configuration options using the client. - -[source,csharp] ----- -var settings= new ElasticsearchClientSettings() - .DefaultMappingFor(i => i - .IndexName("my-projects") - .IdProperty(p => p.Name) - ) - .EnableDebugMode() - .PrettyJson() - .RequestTimeout(TimeSpan.FromMinutes(2)); - -var client = new ElasticsearchClient(settings); ----- diff --git a/docs/connecting.asciidoc b/docs/connecting.asciidoc deleted file mode 100644 index 13d706ba04d..00000000000 --- a/docs/connecting.asciidoc +++ /dev/null @@ -1,173 +0,0 @@ -[[connecting]] -== Connecting - -This page contains the information you need to create an instance of the .NET -Client for {es} that connects to your {es} cluster. - -It's possible to connect to your {es} cluster via a single node, or by -specifying multiple nodes using a node pool. Using a node pool has a few -advantages over a single node, such as load balancing and cluster failover -support. The client provides convenient configuration options to connect to an -Elastic Cloud deployment. - -IMPORTANT: Client applications should create a single instance of -`ElasticsearchClient` that is used throughout your application for its entire -lifetime. Internally the client manages and maintains HTTP connections to nodes, -reusing them to optimize performance. If you use a dependency injection -container for your application, the client instance should be registered with a -singleton lifetime. - -[discrete] -[[cloud-deployment]] -=== Connecting to a cloud deployment - -https://www.elastic.co/guide/en/cloud/current/ec-getting-started.html[Elastic Cloud] -is the easiest way to get started with {es}. When connecting to Elastic Cloud -with the .NET {es} client you should always use the Cloud ID. You can find this -value within the "Manage Deployment" page after you've created a cluster -(look in the top-left if you're in Kibana). - -We recommend using a Cloud ID whenever possible because your client will be -automatically configured for optimal use with Elastic Cloud, including HTTPS and -HTTP compression. - -Connecting to an Elasticsearch Service deployment is achieved by providing the -unique Cloud ID for your deployment when configuring the `ElasticsearchClient` -instance. You also require suitable credentials, either a username and password or -an API key that your application uses to authenticate with your deployment. - -As a security best practice, it is recommended to create a dedicated API key per -application, with permissions limited to only those required for any API calls -the application is authorized to make. - -The following snippet demonstrates how to create a client instance that connects to -an {es} deployment in the cloud. - -[source,csharp] ----- -using Elastic.Clients.Elasticsearch; -using Elastic.Transport; - -var client = new ElasticsearchClient("", new ApiKey("")); <1> ----- -<1> Replace the placeholder string values above with your cloud ID and the API key -configured for your application to access your deployment. - - -[discrete] -[[single-node]] -=== Connecting to a single node - -Single node configuration is best suited to connections to a multi-node cluster -running behind a load balancer or reverse proxy, which is exposed via a single -URL. It may also be convenient to use a single node during local application -development. If the URL represents a single {es} node, be aware that this offers -no resiliency should the server be unreachable or unresponsive. - -By default, security features such as authentication and TLS are enabled on {es} -clusters. When you start {es} for the first time, TLS is configured -automatically for the HTTP layer. A CA certificate is generated and stored on -disk which is used to sign the certificates for the HTTP layer of the {es} -cluster. - -In order for the client to establish a connection with the cluster over HTTPS, -the CA certificate must be trusted by the client application. The simplest -choice is to use the hex-encoded SHA-256 fingerprint of the CA certificate. The -CA fingerprint is output to the terminal when you start {es} for the first time. -You'll see a distinct block like the one below in the output from {es} (you may -have to scroll up if it's been a while): - -```sh ----------------------------------------------------------------- --> Elasticsearch security features have been automatically configured! --> Authentication is enabled and cluster connections are encrypted. - --> Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`): - lhQpLELkjkrawaBoaz0Q - --> HTTP CA certificate SHA-256 fingerprint: - a52dd93511e8c6045e21f16654b77c9ee0f34aea26d9f40320b531c474676228 -... ----------------------------------------------------------------- -``` - -Note down the `elastic` user password and HTTP CA fingerprint for the next -sections. - -The CA fingerprint can also be retrieved at any time from a running cluster using -the following command: - -[source,shell] ----- -openssl x509 -fingerprint -sha256 -in config/certs/http_ca.crt ----- - -The command returns the security certificate, including the fingerprint. The -`issuer` should be `Elasticsearch security auto-configuration HTTP CA`. - -[source,shell] ----- -issuer= /CN=Elasticsearch security auto-configuration HTTP CA -SHA256 Fingerprint= ----- - -Visit the -{ref}/configuring-stack-security.html[Start the Elastic Stack with security enabled automatically] -documentation for more information. - -The following snippet shows you how to create a client instance that connects to -your {es} cluster via a single node, using the CA fingerprint: - -[source,csharp] ----- -using Elastic.Clients.Elasticsearch; -using Elastic.Transport; - -var settings = new ElasticsearchClientSettings(new Uri("https://localhost:9200")) - .CertificateFingerprint("") - .Authentication(new BasicAuthentication("", "")); - -var client = new ElasticsearchClient(settings); ----- - -The preceding snippet demonstrates configuring the client to authenticate by -providing a username and password with basic authentication. If preferred, you -may also use `ApiKey` authentication as shown in the cloud connection example. - -[discrete] -[[multiple-nodes]] -=== Connecting to multiple nodes using a node pool - -To provide resiliency, you should configure multiple nodes for your cluster to -which the client attempts to communicate. By default, the client cycles through -nodes for each request in a round robin fashion. The client also tracks -unhealthy nodes and avoids sending requests to them until they become healthy. - -This configuration is best suited to connect to a known small sized cluster, -where you do not require sniffing to detect the cluster topology. - -The following snippet shows you how to connect to multiple nodes by using a -static node pool: - -[source,csharp] ----- -using Elastic.Clients.Elasticsearch; -using Elastic.Transport; - -var nodes = new Uri[] -{ - new Uri("https://myserver1:9200"), - new Uri("https://myserver2:9200"), - new Uri("https://myserver3:9200") -}; - -var pool = new StaticNodePool(nodes); - -var settings = new ElasticsearchClientSettings(pool) - .CertificateFingerprint("") - .Authentication(new ApiKey("")); - -var client = new ElasticsearchClient(settings); ----- - - diff --git a/docs/docset.yml b/docs/docset.yml new file mode 100644 index 00000000000..6d9cfbdd2aa --- /dev/null +++ b/docs/docset.yml @@ -0,0 +1,487 @@ +project: '.NET client' +cross_links: + - apm-agent-dotnet + - elasticsearch +toc: + - toc: reference + - toc: release-notes +subs: + ref: "https://www.elastic.co/guide/en/elasticsearch/reference/current" + ref-bare: "https://www.elastic.co/guide/en/elasticsearch/reference" + ref-8x: "https://www.elastic.co/guide/en/elasticsearch/reference/8.1" + ref-80: "https://www.elastic.co/guide/en/elasticsearch/reference/8.0" + ref-7x: "https://www.elastic.co/guide/en/elasticsearch/reference/7.17" + ref-70: "https://www.elastic.co/guide/en/elasticsearch/reference/7.0" + ref-60: "https://www.elastic.co/guide/en/elasticsearch/reference/6.0" + ref-64: "https://www.elastic.co/guide/en/elasticsearch/reference/6.4" + xpack-ref: "https://www.elastic.co/guide/en/x-pack/6.2" + logstash-ref: "https://www.elastic.co/guide/en/logstash/current" + kibana-ref: "https://www.elastic.co/guide/en/kibana/current" + kibana-ref-all: "https://www.elastic.co/guide/en/kibana" + beats-ref-root: "https://www.elastic.co/guide/en/beats" + beats-ref: "https://www.elastic.co/guide/en/beats/libbeat/current" + beats-ref-60: "https://www.elastic.co/guide/en/beats/libbeat/6.0" + beats-ref-63: "https://www.elastic.co/guide/en/beats/libbeat/6.3" + beats-devguide: "https://www.elastic.co/guide/en/beats/devguide/current" + auditbeat-ref: "https://www.elastic.co/guide/en/beats/auditbeat/current" + packetbeat-ref: "https://www.elastic.co/guide/en/beats/packetbeat/current" + metricbeat-ref: "https://www.elastic.co/guide/en/beats/metricbeat/current" + filebeat-ref: "https://www.elastic.co/guide/en/beats/filebeat/current" + functionbeat-ref: "https://www.elastic.co/guide/en/beats/functionbeat/current" + winlogbeat-ref: "https://www.elastic.co/guide/en/beats/winlogbeat/current" + heartbeat-ref: "https://www.elastic.co/guide/en/beats/heartbeat/current" + journalbeat-ref: "https://www.elastic.co/guide/en/beats/journalbeat/current" + ingest-guide: "https://www.elastic.co/guide/en/ingest/current" + fleet-guide: "https://www.elastic.co/guide/en/fleet/current" + apm-guide-ref: "https://www.elastic.co/guide/en/apm/guide/current" + apm-guide-7x: "https://www.elastic.co/guide/en/apm/guide/7.17" + apm-app-ref: "https://www.elastic.co/guide/en/kibana/current" + apm-agents-ref: "https://www.elastic.co/guide/en/apm/agent" + apm-android-ref: "https://www.elastic.co/guide/en/apm/agent/android/current" + apm-py-ref: "https://www.elastic.co/guide/en/apm/agent/python/current" + apm-py-ref-3x: "https://www.elastic.co/guide/en/apm/agent/python/3.x" + apm-node-ref-index: "https://www.elastic.co/guide/en/apm/agent/nodejs" + apm-node-ref: "https://www.elastic.co/guide/en/apm/agent/nodejs/current" + apm-node-ref-1x: "https://www.elastic.co/guide/en/apm/agent/nodejs/1.x" + apm-rum-ref: "https://www.elastic.co/guide/en/apm/agent/rum-js/current" + apm-ruby-ref: "https://www.elastic.co/guide/en/apm/agent/ruby/current" + apm-java-ref: "https://www.elastic.co/guide/en/apm/agent/java/current" + apm-go-ref: "https://www.elastic.co/guide/en/apm/agent/go/current" + apm-dotnet-ref: "https://www.elastic.co/guide/en/apm/agent/dotnet/current" + apm-php-ref: "https://www.elastic.co/guide/en/apm/agent/php/current" + apm-ios-ref: "https://www.elastic.co/guide/en/apm/agent/swift/current" + apm-lambda-ref: "https://www.elastic.co/guide/en/apm/lambda/current" + apm-attacher-ref: "https://www.elastic.co/guide/en/apm/attacher/current" + docker-logging-ref: "https://www.elastic.co/guide/en/beats/loggingplugin/current" + esf-ref: "https://www.elastic.co/guide/en/esf/current" + kinesis-firehose-ref: "https://www.elastic.co/guide/en/kinesis/{{kinesis_version}}" + estc-welcome-current: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current" + estc-welcome: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current" + estc-welcome-all: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions" + hadoop-ref: "https://www.elastic.co/guide/en/elasticsearch/hadoop/current" + stack-ref: "https://www.elastic.co/guide/en/elastic-stack/current" + stack-ref-67: "https://www.elastic.co/guide/en/elastic-stack/6.7" + stack-ref-68: "https://www.elastic.co/guide/en/elastic-stack/6.8" + stack-ref-70: "https://www.elastic.co/guide/en/elastic-stack/7.0" + stack-ref-80: "https://www.elastic.co/guide/en/elastic-stack/8.0" + stack-ov: "https://www.elastic.co/guide/en/elastic-stack-overview/current" + stack-gs: "https://www.elastic.co/guide/en/elastic-stack-get-started/current" + stack-gs-current: "https://www.elastic.co/guide/en/elastic-stack-get-started/current" + javaclient: "https://www.elastic.co/guide/en/elasticsearch/client/java-api/current" + java-api-client: "https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current" + java-rest: "https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current" + jsclient: "https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current" + jsclient-current: "https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current" + es-ruby-client: "https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current" + es-dotnet-client: "https://www.elastic.co/guide/en/elasticsearch/client/net-api/current" + es-php-client: "https://www.elastic.co/guide/en/elasticsearch/client/php-api/current" + es-python-client: "https://www.elastic.co/guide/en/elasticsearch/client/python-api/current" + defguide: "https://www.elastic.co/guide/en/elasticsearch/guide/2.x" + painless: "https://www.elastic.co/guide/en/elasticsearch/painless/current" + plugins: "https://www.elastic.co/guide/en/elasticsearch/plugins/current" + plugins-8x: "https://www.elastic.co/guide/en/elasticsearch/plugins/8.1" + plugins-7x: "https://www.elastic.co/guide/en/elasticsearch/plugins/7.17" + plugins-6x: "https://www.elastic.co/guide/en/elasticsearch/plugins/6.8" + glossary: "https://www.elastic.co/guide/en/elastic-stack-glossary/current" + upgrade_guide: "https://www.elastic.co/products/upgrade_guide" + blog-ref: "https://www.elastic.co/blog/" + curator-ref: "https://www.elastic.co/guide/en/elasticsearch/client/curator/current" + curator-ref-current: "https://www.elastic.co/guide/en/elasticsearch/client/curator/current" + metrics-ref: "https://www.elastic.co/guide/en/metrics/current" + metrics-guide: "https://www.elastic.co/guide/en/metrics/guide/current" + logs-ref: "https://www.elastic.co/guide/en/logs/current" + logs-guide: "https://www.elastic.co/guide/en/logs/guide/current" + uptime-guide: "https://www.elastic.co/guide/en/uptime/current" + observability-guide: "https://www.elastic.co/guide/en/observability/current" + observability-guide-all: "https://www.elastic.co/guide/en/observability" + siem-guide: "https://www.elastic.co/guide/en/siem/guide/current" + security-guide: "https://www.elastic.co/guide/en/security/current" + security-guide-all: "https://www.elastic.co/guide/en/security" + endpoint-guide: "https://www.elastic.co/guide/en/endpoint/current" + sql-odbc: "https://www.elastic.co/guide/en/elasticsearch/sql-odbc/current" + ecs-ref: "https://www.elastic.co/guide/en/ecs/current" + ecs-logging-ref: "https://www.elastic.co/guide/en/ecs-logging/overview/current" + ecs-logging-go-logrus-ref: "https://www.elastic.co/guide/en/ecs-logging/go-logrus/current" + ecs-logging-go-zap-ref: "https://www.elastic.co/guide/en/ecs-logging/go-zap/current" + ecs-logging-go-zerolog-ref: "https://www.elastic.co/guide/en/ecs-logging/go-zap/current" + ecs-logging-java-ref: "https://www.elastic.co/guide/en/ecs-logging/java/current" + ecs-logging-dotnet-ref: "https://www.elastic.co/guide/en/ecs-logging/dotnet/current" + ecs-logging-nodejs-ref: "https://www.elastic.co/guide/en/ecs-logging/nodejs/current" + ecs-logging-php-ref: "https://www.elastic.co/guide/en/ecs-logging/php/current" + ecs-logging-python-ref: "https://www.elastic.co/guide/en/ecs-logging/python/current" + ecs-logging-ruby-ref: "https://www.elastic.co/guide/en/ecs-logging/ruby/current" + ml-docs: "https://www.elastic.co/guide/en/machine-learning/current" + eland-docs: "https://www.elastic.co/guide/en/elasticsearch/client/eland/current" + eql-ref: "https://eql.readthedocs.io/en/latest/query-guide" + extendtrial: "https://www.elastic.co/trialextension" + wikipedia: "https://en.wikipedia.org/wiki" + forum: "https://discuss.elastic.co/" + xpack-forum: "https://discuss.elastic.co/c/50-x-pack" + security-forum: "https://discuss.elastic.co/c/x-pack/shield" + watcher-forum: "https://discuss.elastic.co/c/x-pack/watcher" + monitoring-forum: "https://discuss.elastic.co/c/x-pack/marvel" + graph-forum: "https://discuss.elastic.co/c/x-pack/graph" + apm-forum: "https://discuss.elastic.co/c/apm" + enterprise-search-ref: "https://www.elastic.co/guide/en/enterprise-search/current" + app-search-ref: "https://www.elastic.co/guide/en/app-search/current" + workplace-search-ref: "https://www.elastic.co/guide/en/workplace-search/current" + enterprise-search-node-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/enterprise-search-node/current" + enterprise-search-php-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/php/current" + enterprise-search-python-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/python/current" + enterprise-search-ruby-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/ruby/current" + elastic-maps-service: "https://maps.elastic.co" + integrations-docs: "https://docs.elastic.co/en/integrations" + integrations-devguide: "https://www.elastic.co/guide/en/integrations-developer/current" + time-units: "https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#time-units" + byte-units: "https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#byte-units" + apm-py-ref-v: "https://www.elastic.co/guide/en/apm/agent/python/current" + apm-node-ref-v: "https://www.elastic.co/guide/en/apm/agent/nodejs/current" + apm-rum-ref-v: "https://www.elastic.co/guide/en/apm/agent/rum-js/current" + apm-ruby-ref-v: "https://www.elastic.co/guide/en/apm/agent/ruby/current" + apm-java-ref-v: "https://www.elastic.co/guide/en/apm/agent/java/current" + apm-go-ref-v: "https://www.elastic.co/guide/en/apm/agent/go/current" + apm-ios-ref-v: "https://www.elastic.co/guide/en/apm/agent/swift/current" + apm-dotnet-ref-v: "https://www.elastic.co/guide/en/apm/agent/dotnet/current" + apm-php-ref-v: "https://www.elastic.co/guide/en/apm/agent/php/current" + ecloud: "Elastic Cloud" + esf: "Elastic Serverless Forwarder" + ess: "Elasticsearch Service" + ece: "Elastic Cloud Enterprise" + eck: "Elastic Cloud on Kubernetes" + serverless-full: "Elastic Cloud Serverless" + serverless-short: "Serverless" + es-serverless: "Elasticsearch Serverless" + es3: "Elasticsearch Serverless" + obs-serverless: "Elastic Observability Serverless" + sec-serverless: "Elastic Security Serverless" + serverless-docs: "https://docs.elastic.co/serverless" + cloud: "https://www.elastic.co/guide/en/cloud/current" + ess-utm-params: "?page=docs&placement=docs-body" + ess-baymax: "?page=docs&placement=docs-body" + ess-trial: "https://cloud.elastic.co/registration?page=docs&placement=docs-body" + ess-product: "https://www.elastic.co/cloud/elasticsearch-service?page=docs&placement=docs-body" + ess-console: "https://cloud.elastic.co?page=docs&placement=docs-body" + ess-console-name: "Elasticsearch Service Console" + ess-deployments: "https://cloud.elastic.co/deployments?page=docs&placement=docs-body" + ece-ref: "https://www.elastic.co/guide/en/cloud-enterprise/current" + eck-ref: "https://www.elastic.co/guide/en/cloud-on-k8s/current" + ess-leadin: "You can run Elasticsearch on your own hardware or use our hosted Elasticsearch Service that is available on AWS, GCP, and Azure. https://cloud.elastic.co/registration{ess-utm-params}[Try the Elasticsearch Service for free]." + ess-leadin-short: "Our hosted Elasticsearch Service is available on AWS, GCP, and Azure, and you can https://cloud.elastic.co/registration{ess-utm-params}[try it for free]." + ess-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud.svg[link=\"https://cloud.elastic.co/registration{ess-utm-params}\", title=\"Supported on Elasticsearch Service\"]" + ece-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud_ece.svg[link=\"https://cloud.elastic.co/registration{ess-utm-params}\", title=\"Supported on Elastic Cloud Enterprise\"]" + cloud-only: "This feature is designed for indirect use by https://cloud.elastic.co/registration{ess-utm-params}[Elasticsearch Service], https://www.elastic.co/guide/en/cloud-enterprise/{ece-version-link}[Elastic Cloud Enterprise], and https://www.elastic.co/guide/en/cloud-on-k8s/current[Elastic Cloud on Kubernetes]. Direct use is not supported." + ess-setting-change: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud.svg[link=\"{ess-trial}\", title=\"Supported on {ess}\"] indicates a change to a supported https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html[user setting] for Elasticsearch Service." + ess-skip-section: "If you use Elasticsearch Service, skip this section. Elasticsearch Service handles these changes for you." + api-cloud: "https://www.elastic.co/docs/api/doc/cloud" + api-ece: "https://www.elastic.co/docs/api/doc/cloud-enterprise" + api-kibana-serverless: "https://www.elastic.co/docs/api/doc/serverless" + es-feature-flag: "This feature is in development and not yet available for use. This documentation is provided for informational purposes only." + es-ref-dir: "'{{elasticsearch-root}}/docs/reference'" + apm-app: "APM app" + uptime-app: "Uptime app" + synthetics-app: "Synthetics app" + logs-app: "Logs app" + metrics-app: "Metrics app" + infrastructure-app: "Infrastructure app" + siem-app: "SIEM app" + security-app: "Elastic Security app" + ml-app: "Machine Learning" + dev-tools-app: "Dev Tools" + ingest-manager-app: "Ingest Manager" + stack-manage-app: "Stack Management" + stack-monitor-app: "Stack Monitoring" + alerts-ui: "Alerts and Actions" + rules-ui: "Rules" + rac-ui: "Rules and Connectors" + connectors-ui: "Connectors" + connectors-feature: "Actions and Connectors" + stack-rules-feature: "Stack Rules" + user-experience: "User Experience" + ems: "Elastic Maps Service" + ems-init: "EMS" + hosted-ems: "Elastic Maps Server" + ipm-app: "Index Pattern Management" + ingest-pipelines: "ingest pipelines" + ingest-pipelines-app: "Ingest Pipelines" + ingest-pipelines-cap: "Ingest pipelines" + ls-pipelines: "Logstash pipelines" + ls-pipelines-app: "Logstash Pipelines" + maint-windows: "maintenance windows" + maint-windows-app: "Maintenance Windows" + maint-windows-cap: "Maintenance windows" + custom-roles-app: "Custom Roles" + data-source: "data view" + data-sources: "data views" + data-source-caps: "Data View" + data-sources-caps: "Data Views" + data-source-cap: "Data view" + data-sources-cap: "Data views" + project-settings: "Project settings" + manage-app: "Management" + index-manage-app: "Index Management" + data-views-app: "Data Views" + rules-app: "Rules" + saved-objects-app: "Saved Objects" + tags-app: "Tags" + api-keys-app: "API keys" + transforms-app: "Transforms" + connectors-app: "Connectors" + files-app: "Files" + reports-app: "Reports" + maps-app: "Maps" + alerts-app: "Alerts" + crawler: "Enterprise Search web crawler" + ents: "Enterprise Search" + app-search-crawler: "App Search web crawler" + agent: "Elastic Agent" + agents: "Elastic Agents" + fleet: "Fleet" + fleet-server: "Fleet Server" + integrations-server: "Integrations Server" + ingest-manager: "Ingest Manager" + ingest-management: "ingest management" + package-manager: "Elastic Package Manager" + integrations: "Integrations" + package-registry: "Elastic Package Registry" + artifact-registry: "Elastic Artifact Registry" + aws: "AWS" + stack: "Elastic Stack" + xpack: "X-Pack" + es: "Elasticsearch" + kib: "Kibana" + esms: "Elastic Stack Monitoring Service" + esms-init: "ESMS" + ls: "Logstash" + beats: "Beats" + auditbeat: "Auditbeat" + filebeat: "Filebeat" + heartbeat: "Heartbeat" + metricbeat: "Metricbeat" + packetbeat: "Packetbeat" + winlogbeat: "Winlogbeat" + functionbeat: "Functionbeat" + journalbeat: "Journalbeat" + es-sql: "Elasticsearch SQL" + esql: "ES|QL" + elastic-agent: "Elastic Agent" + k8s: "Kubernetes" + log-driver-long: "Elastic Logging Plugin for Docker" + security: "X-Pack security" + security-features: "security features" + operator-feature: "operator privileges feature" + es-security-features: "Elasticsearch security features" + stack-security-features: "Elastic Stack security features" + endpoint-sec: "Endpoint Security" + endpoint-cloud-sec: "Endpoint and Cloud Security" + elastic-defend: "Elastic Defend" + elastic-sec: "Elastic Security" + elastic-endpoint: "Elastic Endpoint" + swimlane: "Swimlane" + sn: "ServiceNow" + sn-itsm: "ServiceNow ITSM" + sn-itom: "ServiceNow ITOM" + sn-sir: "ServiceNow SecOps" + jira: "Jira" + ibm-r: "IBM Resilient" + webhook: "Webhook" + webhook-cm: "Webhook - Case Management" + opsgenie: "Opsgenie" + bedrock: "Amazon Bedrock" + gemini: "Google Gemini" + hive: "TheHive" + monitoring: "X-Pack monitoring" + monitor-features: "monitoring features" + stack-monitor-features: "Elastic Stack monitoring features" + watcher: "Watcher" + alert-features: "alerting features" + reporting: "X-Pack reporting" + report-features: "reporting features" + graph: "X-Pack graph" + graph-features: "graph analytics features" + searchprofiler: "Search Profiler" + xpackml: "X-Pack machine learning" + ml: "machine learning" + ml-cap: "Machine learning" + ml-init: "ML" + ml-features: "machine learning features" + stack-ml-features: "Elastic Stack machine learning features" + ccr: "cross-cluster replication" + ccr-cap: "Cross-cluster replication" + ccr-init: "CCR" + ccs: "cross-cluster search" + ccs-cap: "Cross-cluster search" + ccs-init: "CCS" + ilm: "index lifecycle management" + ilm-cap: "Index lifecycle management" + ilm-init: "ILM" + dlm: "data lifecycle management" + dlm-cap: "Data lifecycle management" + dlm-init: "DLM" + search-snap: "searchable snapshot" + search-snaps: "searchable snapshots" + search-snaps-cap: "Searchable snapshots" + slm: "snapshot lifecycle management" + slm-cap: "Snapshot lifecycle management" + slm-init: "SLM" + rollup-features: "data rollup features" + ipm: "index pattern management" + ipm-cap: "Index pattern" + rollup: "rollup" + rollup-cap: "Rollup" + rollups: "rollups" + rollups-cap: "Rollups" + rollup-job: "rollup job" + rollup-jobs: "rollup jobs" + rollup-jobs-cap: "Rollup jobs" + dfeed: "datafeed" + dfeeds: "datafeeds" + dfeed-cap: "Datafeed" + dfeeds-cap: "Datafeeds" + ml-jobs: "machine learning jobs" + ml-jobs-cap: "Machine learning jobs" + anomaly-detect: "anomaly detection" + anomaly-detect-cap: "Anomaly detection" + anomaly-job: "anomaly detection job" + anomaly-jobs: "anomaly detection jobs" + anomaly-jobs-cap: "Anomaly detection jobs" + dataframe: "data frame" + dataframes: "data frames" + dataframe-cap: "Data frame" + dataframes-cap: "Data frames" + watcher-transform: "payload transform" + watcher-transforms: "payload transforms" + watcher-transform-cap: "Payload transform" + watcher-transforms-cap: "Payload transforms" + transform: "transform" + transforms: "transforms" + transform-cap: "Transform" + transforms-cap: "Transforms" + dataframe-transform: "transform" + dataframe-transform-cap: "Transform" + dataframe-transforms: "transforms" + dataframe-transforms-cap: "Transforms" + dfanalytics-cap: "Data frame analytics" + dfanalytics: "data frame analytics" + dataframe-analytics-config: "'{dataframe} analytics config'" + dfanalytics-job: "'{dataframe} analytics job'" + dfanalytics-jobs: "'{dataframe} analytics jobs'" + dfanalytics-jobs-cap: "'{dataframe-cap} analytics jobs'" + cdataframe: "continuous data frame" + cdataframes: "continuous data frames" + cdataframe-cap: "Continuous data frame" + cdataframes-cap: "Continuous data frames" + cdataframe-transform: "continuous transform" + cdataframe-transforms: "continuous transforms" + cdataframe-transforms-cap: "Continuous transforms" + ctransform: "continuous transform" + ctransform-cap: "Continuous transform" + ctransforms: "continuous transforms" + ctransforms-cap: "Continuous transforms" + oldetection: "outlier detection" + oldetection-cap: "Outlier detection" + olscore: "outlier score" + olscores: "outlier scores" + fiscore: "feature influence score" + evaluatedf-api: "evaluate {dataframe} analytics API" + evaluatedf-api-cap: "Evaluate {dataframe} analytics API" + binarysc: "binary soft classification" + binarysc-cap: "Binary soft classification" + regression: "regression" + regression-cap: "Regression" + reganalysis: "regression analysis" + reganalysis-cap: "Regression analysis" + depvar: "dependent variable" + feature-var: "feature variable" + feature-vars: "feature variables" + feature-vars-cap: "Feature variables" + classification: "classification" + classification-cap: "Classification" + classanalysis: "classification analysis" + classanalysis-cap: "Classification analysis" + infer-cap: "Inference" + infer: "inference" + lang-ident-cap: "Language identification" + lang-ident: "language identification" + data-viz: "Data Visualizer" + file-data-viz: "File Data Visualizer" + feat-imp: "feature importance" + feat-imp-cap: "Feature importance" + nlp: "natural language processing" + nlp-cap: "Natural language processing" + apm-agent: "APM agent" + apm-go-agent: "Elastic APM Go agent" + apm-go-agents: "Elastic APM Go agents" + apm-ios-agent: "Elastic APM iOS agent" + apm-ios-agents: "Elastic APM iOS agents" + apm-java-agent: "Elastic APM Java agent" + apm-java-agents: "Elastic APM Java agents" + apm-dotnet-agent: "Elastic APM .NET agent" + apm-dotnet-agents: "Elastic APM .NET agents" + apm-node-agent: "Elastic APM Node.js agent" + apm-node-agents: "Elastic APM Node.js agents" + apm-php-agent: "Elastic APM PHP agent" + apm-php-agents: "Elastic APM PHP agents" + apm-py-agent: "Elastic APM Python agent" + apm-py-agents: "Elastic APM Python agents" + apm-ruby-agent: "Elastic APM Ruby agent" + apm-ruby-agents: "Elastic APM Ruby agents" + apm-rum-agent: "Elastic APM Real User Monitoring (RUM) JavaScript agent" + apm-rum-agents: "Elastic APM RUM JavaScript agents" + apm-lambda-ext: "Elastic APM AWS Lambda extension" + project-monitors: "project monitors" + project-monitors-cap: "Project monitors" + private-location: "Private Location" + private-locations: "Private Locations" + pwd: "YOUR_PASSWORD" + esh: "ES-Hadoop" + default-dist: "default distribution" + oss-dist: "OSS-only distribution" + observability: "Observability" + api-request-title: "Request" + api-prereq-title: "Prerequisites" + api-description-title: "Description" + api-path-parms-title: "Path parameters" + api-query-parms-title: "Query parameters" + api-request-body-title: "Request body" + api-response-codes-title: "Response codes" + api-response-body-title: "Response body" + api-example-title: "Example" + api-examples-title: "Examples" + api-definitions-title: "Properties" + multi-arg: "†footnoteref:[multi-arg,This parameter accepts multiple arguments.]" + multi-arg-ref: "†footnoteref:[multi-arg]" + yes-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png[Yes,20,15]" + no-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png[No,20,15]" + es-repo: "https://github.com/elastic/elasticsearch/" + es-issue: "https://github.com/elastic/elasticsearch/issues/" + es-pull: "https://github.com/elastic/elasticsearch/pull/" + es-commit: "https://github.com/elastic/elasticsearch/commit/" + kib-repo: "https://github.com/elastic/kibana/" + kib-issue: "https://github.com/elastic/kibana/issues/" + kibana-issue: "'{kib-repo}issues/'" + kib-pull: "https://github.com/elastic/kibana/pull/" + kibana-pull: "'{kib-repo}pull/'" + kib-commit: "https://github.com/elastic/kibana/commit/" + ml-repo: "https://github.com/elastic/ml-cpp/" + ml-issue: "https://github.com/elastic/ml-cpp/issues/" + ml-pull: "https://github.com/elastic/ml-cpp/pull/" + ml-commit: "https://github.com/elastic/ml-cpp/commit/" + apm-repo: "https://github.com/elastic/apm-server/" + apm-issue: "https://github.com/elastic/apm-server/issues/" + apm-pull: "https://github.com/elastic/apm-server/pull/" + kibana-blob: "https://github.com/elastic/kibana/blob/current/" + apm-get-started-ref: "https://www.elastic.co/guide/en/apm/get-started/current" + apm-server-ref: "https://www.elastic.co/guide/en/apm/server/current" + apm-server-ref-v: "https://www.elastic.co/guide/en/apm/server/current" + apm-server-ref-m: "https://www.elastic.co/guide/en/apm/server/master" + apm-server-ref-62: "https://www.elastic.co/guide/en/apm/server/6.2" + apm-server-ref-64: "https://www.elastic.co/guide/en/apm/server/6.4" + apm-server-ref-70: "https://www.elastic.co/guide/en/apm/server/7.0" + apm-overview-ref-v: "https://www.elastic.co/guide/en/apm/get-started/current" + apm-overview-ref-70: "https://www.elastic.co/guide/en/apm/get-started/7.0" + apm-overview-ref-m: "https://www.elastic.co/guide/en/apm/get-started/master" + infra-guide: "https://www.elastic.co/guide/en/infrastructure/guide/current" + a-data-source: "a data view" + icon-bug: "pass:[]" + icon-checkInCircleFilled: "pass:[]" + icon-warningFilled: "pass:[]" diff --git a/docs/getting-started.asciidoc b/docs/getting-started.asciidoc deleted file mode 100644 index aad525c2b1b..00000000000 --- a/docs/getting-started.asciidoc +++ /dev/null @@ -1,161 +0,0 @@ -[[getting-started-net]] -== Getting started - -This page guides you through the installation process of the .NET client, shows -you how to instantiate the client, and how to perform basic Elasticsearch -operations with it. - -[discrete] -=== Requirements - -* .NET Core, .NET 5+ or .NET Framework (4.6.1 and higher). - -[discrete] -=== Installation - -To install the latest version of the client for SDK style projects, run the following command: - -[source,shell] --------------------------- -dotnet add package Elastic.Clients.Elasticsearch --------------------------- - -Refer to the <> page to learn more. - - -[discrete] -=== Connecting - -You can connect to the Elastic Cloud using an API key and the Elasticsearch -endpoint. - -[source,net] ----- -var client = new ElasticsearchClient("", new ApiKey("")); ----- - -Your Elasticsearch endpoint can be found on the **My deployment** page of your -deployment: - -image::images/es-endpoint.jpg[alt="Finding Elasticsearch endpoint",align="center"] - -You can generate an API key on the **Management** page under Security. - -image::images/create-api-key.png[alt="Create API key",align="center"] - -For other connection options, refer to the <> section. - - -[discrete] -=== Operations - -Time to use Elasticsearch! This section walks you through the basic, and most -important, operations of Elasticsearch. For more operations and more advanced -examples, refer to the <> page. - - -[discrete] -==== Creating an index - -This is how you create the `my_index` index: - -[source,net] ----- -var response = await client.Indices.CreateAsync("my_index"); ----- - - -[discrete] -==== Indexing documents - -This is a simple way of indexing a document: - -[source,net] ----- -var doc = new MyDoc -{ - Id = 1, - User = "flobernd", - Message = "Trying out the client, so far so good?" -}; - -var response = await client.IndexAsync(doc, "my_index"); ----- - - -[discrete] -==== Getting documents - -You can get documents by using the following code: - -[source,net] ----- -var response = await client.GetAsync(id, idx => idx.Index("my_index")); - -if (response.IsValidResponse) -{ - var doc = response.Source; -} ----- - - -[discrete] -==== Searching documents - -This is how you can create a single match query with the .NET client: - -[source,net] ----- -var response = await client.SearchAsync(s => s - .Index("my_index") - .From(0) - .Size(10) - .Query(q => q - .Term(t => t.User, "flobernd") - ) -); - -if (response.IsValidResponse) -{ - var doc = response.Documents.FirstOrDefault(); -} ----- - - -[discrete] -==== Updating documents - -This is how you can update a document, for example to add a new field: - -[source,net] ----- -doc.Message = "This is a new message"; - -var response = await client.UpdateAsync("my_index", 1, u => u - .Doc(doc)); ----- - - -[discrete] -==== Deleting documents - -[source,net] ----- -var response = await client.DeleteAsync("my_index", 1); ----- - - -[discrete] -==== Deleting an index - -[source,net] ----- -var response = await client.Indices.DeleteAsync("my_index"); ----- - - -[discrete] -== Further reading - -* Refer to the <> page to learn more about how to use the -client the most efficiently. \ No newline at end of file diff --git a/docs/index.asciidoc b/docs/index.asciidoc deleted file mode 100644 index b1202f34de2..00000000000 --- a/docs/index.asciidoc +++ /dev/null @@ -1,33 +0,0 @@ -[[elasticsearch-net-reference]] -= Elasticsearch .NET Client - -include::{asciidoc-dir}/../../shared/versions/stack/{source_branch}.asciidoc[] -include::{asciidoc-dir}/../../shared/attributes.asciidoc[] - -:doc-tests-src: {docdir}/../tests/Tests/Documentation -:net-client: Elasticsearch .NET Client -:latest-version: 8.15.8 - -:es-docs: https://www.elastic.co/guide/en/elasticsearch/reference/{branch} - -include::intro.asciidoc[] - -include::getting-started.asciidoc[] - -include::install.asciidoc[] - -include::connecting.asciidoc[] - -include::configuration.asciidoc[] - -include::client-concepts/client-concepts.asciidoc[] - -include::usage/index.asciidoc[] - -include::migration-guide.asciidoc[] - -include::troubleshooting.asciidoc[] - -include::redirects.asciidoc[] - -include::release-notes/release-notes.asciidoc[] \ No newline at end of file diff --git a/docs/install.asciidoc b/docs/install.asciidoc deleted file mode 100644 index 09a383cda5a..00000000000 --- a/docs/install.asciidoc +++ /dev/null @@ -1,95 +0,0 @@ -[[installation]] -== Installation - -This page shows you how to install the .NET client for {es}. - -IMPORTANT: The v8 client for .NET does not have complete feature parity with -the v7 `NEST` client. It may not be suitable for for all applications until -additional endpoints and features are supported. We therefore recommend you thoroughly -review our <> before attempting to migrate -existing applications to the `Elastic.Clients.Elasticsearch` library. -Until the new client supports all endpoints and features your application requires, -you may continue to use the 7.17.x https://www.nuget.org/packages/NEST[NEST] client -to communicate with v8 Elasticsearch servers using compatibility mode. Refer to the -https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html[Connecting to Elasticsearch v8.x using the v7.17.x client documentation] -for guidance on configuring the 7.17.x client. - -[discrete] -[[dot-net-client]] -=== Installing the .NET client - -For SDK style projects, you can install the {es} client by running the following -.NET CLI command in your terminal: - -[source,text] ----- -dotnet add package Elastic.Clients.Elasticsearch ----- - -This command adds a package reference to your project (csproj) file for the -latest stable version of the client. - -If you prefer, you may also manually add a package reference inside your project -file: - -[source,shell] ----- - ----- -_NOTE: The version number should reflect the latest published version from -https://www.nuget.org/packages/Elastic.Clients.Elasticsearch[NuGet.org]. To install -a different version, modify the version as necessary._ - -For Visual Studio users, the .NET client can also be installed from the Package -Manager Console inside Visual Studio using the following command: - -[source,shell] ----- -Install-Package Elastic.Clients.Elasticsearch ----- - -Alternatively, search for `Elastic.Clients.Elasticsearch` in the NuGet Package -Manager UI. - -To learn how to connect the {es} client, refer to the <> section. - -[discrete] -[[compatibility]] -=== Compatibility - -The {es} client is compatible with currently maintained .NET runtime versions. -Compatibility with End of Life (EOL) .NET runtimes is not guaranteed or -supported. - -Language clients are forward compatible; meaning that the clients support -communicating with greater or equal minor versions of {es} without breaking. It -does not mean that the clients automatically support new features of newer -{es} versions; it is only possible after a release of a new client version. For -example, a 8.12 client version won't automatically support the new features of -the 8.13 version of {es}, the 8.13 client version is required for that. {es} -language clients are only backwards compatible with default distributions and -without guarantees made. - -|=== -| Elasticsearch Version | Elasticsearch-NET Branch | Supported - -| main | main | -| 8.x | 8.x | 8.x -| 7.x | 7.x | 7.17 -|=== - -Refer to the https://www.elastic.co/support/eol[end-of-life policy] for more -information. - -[discrete] -[[ci-feed]] -=== CI feed - -We publish CI builds of our client packages, including the latest -unreleased features. If you want to experiment with the latest bits, you -can add the CI feed to your list of NuGet package sources. - -Feed URL: https://f.feedz.io/elastic/all/nuget/index.json - -We do not recommend using CI builds for production applications as they are not -formally supported until they are released. \ No newline at end of file diff --git a/docs/intro.asciidoc b/docs/intro.asciidoc deleted file mode 100644 index d638ad6acac..00000000000 --- a/docs/intro.asciidoc +++ /dev/null @@ -1,54 +0,0 @@ -:github: https://github.com/elastic/elasticsearch-net - -[[introduction]] -== Introduction - -*Rapidly develop applications with the .NET client for {es}.* - -Designed for .NET application developers, the .NET language client -library provides a strongly typed API and query DSL for interacting with {es}. -The .NET client includes higher-level abstractions, such as -helpers for coordinating bulk indexing and update operations. It also comes with -built-in, configurable cluster failover retry mechanisms. - -The {es} .NET client is available as a https://www.nuget.org/packages/Elastic.Clients.Elasticsearch[NuGet] -package for use with .NET Core, .NET 5+, and .NET Framework (4.6.1 and later) -applications. - -_NOTE: This documentation covers the v8 .NET client for {es}, for use -with {es} 8.x versions. To develop applications targeting {es} v7, use the -https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17[v7 (NEST) client]._ - -[discrete] -[[features]] -=== Features - -* One-to-one mapping with the REST API. -* Strongly typed requests and responses for {es} APIs. -* Fluent API for building requests. -* Query DSL to assist with constructing search queries. -* Helpers for common tasks such as bulk indexing of documents. -* Pluggable serialization of requests and responses based on `System.Text.Json`. -* Diagnostics, auditing, and .NET activity integration. - -The .NET {es} client is built on the Elastic Transport library, which provides: - -* Connection management and load balancing across all available nodes. -* Request retries and dead connections handling. - -[discrete] -=== {es} version compatibility - -Language clients are forward compatible: clients support communicating -with current and later minor versions of {es}. {es} language clients are -backward compatible with default distributions only and without guarantees. - -[discrete] -=== Questions, bugs, comments, feature requests - -To submit a bug report or feature request, use -{github}/issues[GitHub issues]. - -For more general questions and comments, try the community forum -on https://discuss.elastic.co/c/elasticsearch[discuss.elastic.co]. -Mention `.NET` in the title to indicate the discussion topic. \ No newline at end of file diff --git a/docs/migration-guide.asciidoc b/docs/migration-guide.asciidoc deleted file mode 100644 index 1d3ae76032c..00000000000 --- a/docs/migration-guide.asciidoc +++ /dev/null @@ -1,334 +0,0 @@ -[[migration-guide]] -== Migration guide: From NEST v7 to .NET Client v8 - -The following migration guide explains the current state of the client, missing -features, breaking changes and our rationale for some of the design choices we have introduced. - -[discrete] -=== Version 8 is a refresh - -[IMPORTANT] --- -It is important to highlight that v8 of the {net-client} represents -a new start for the client design. It is important to review how this may affect -your code and usage. --- - -Mature code becomes increasingly hard to maintain over time. -Major releases allow us to simplify and better align our language clients with -each other in terms of design. It is crucial to find the right balance -between uniformity across programming languages and the idiomatic concerns of -each language. For .NET, we typically compare and contrast with https://github.com/elastic/elasticsearch-java[Java] and https://github.com/elastic/go-elasticsearch[Go] -to make sure that our approach is equivalent for each of these. We also take -heavy inspiration from Microsoft framework design guidelines and the conventions -of the wider .NET community. - -[discrete] -==== New Elastic.Clients.Elasticsearch NuGet package - -We have shipped the new code-generated client as a -https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/[NuGet package] -with a new root namespace, `Elastic.Clients.Elasticsearch`. -The v8 client is built upon the foundations of the v7 `NEST` client, but there -are changes. By shipping as a new package, the expectation is that migration can -be managed with a phased approach. - -While this is a new package, we have aligned the major version (v8.x.x) with the -supported {es} server version to clearly indicate the client/server compatibility. -The v8 client is designed to work with version 8 of {es}. - -The v7 `NEST` client continues to be supported but will not gain new features or -support for new {es} endpoints. It should be considered deprecated in favour of -the new client. - -[discrete] -==== Limited feature set - -[CAUTION] --- -The version 8 {net-client} does not have feature parity with the previous v7 `NEST` -high-level client. --- - -If a feature you depend on is missing (and not explicitly documented below as a -feature that we do not plan to reintroduce), open https://github.com/elastic/elasticsearch-net/issues/new/choose[an issue] -or comment on a relevant existing issue to highlight your need to us. This will -help us prioritise our roadmap. - -[discrete] -=== Code generation - -Given the size of the {es} API surface today, it is no longer practical -to maintain thousands of types (requests, responses, queries, aggregations, etc.) -by hand. To ensure consistent, accurate, and timely alignment between language -clients and {es}, the 8.x clients, and many of the associated types are now -automatically code-generated from a https://github.com/elastic/elasticsearch-specification[shared specification]. This is a common solution to maintaining alignment between -client and server among SDKs and libraries, such as those for Azure, AWS and the -Google Cloud Platform. - -Code-generation from a specification has inevitably led to some differences -between the existing v7 `NEST` types and those available in the new v7 {net-client}. -For version 8, we generate strictly from the specification, special -casing a few areas to improve usability or to align with language idioms. - -The base type hierarchy for concepts such as `Properties`, `Aggregations` and -`Queries` is no longer present in generated code, as these arbitrary groupings do -not align with concrete concepts of the public server API. These considerations -do not preclude adding syntactic sugar and usability enhancements to types in future -releases on a case-by-case basis. - -[discrete] -=== Elastic.Transport - -The .NET client includes a transport layer responsible for abstracting HTTP -concepts and to provide functionality such as our request pipeline. This -supports round-robin load-balancing of requests to nodes, pinging failed -nodes and sniffing the cluster for node roles. - -In v7, this layer shipped as `Elasticsearch.Net` and was considered our low-level -client which could be used to send and receive raw JSON bytes between the client -and server. - -As part of the work for 8.0.0, we have moved the transport layer out into -a https://www.nuget.org/packages/Elastic.Transport[new dedicated package] and -https://github.com/elastic/elastic-transport-net[repository], named -`Elastic.Transport`. This supports reuse across future clients and allows -consumers with extremely high-performance requirements to build upon this foundation. - -[discrete] -=== System.Text.Json for serialization - -The v7 `NEST` high-level client used an internalized and modified version of -https://github.com/neuecc/Utf8Json[Utf8Json] for request and response -serialization. This was introduced for its performance improvements -over https://www.newtonsoft.com/json[Json.NET], the more common JSON framework at -the time. - -While Utf8Json provides good value, we have identified minor bugs and -performance issues that have required maintenance over time. Some of these -are hard to change without more significant effort. This library is no longer -maintained, and any such changes cannot easily be contributed back to the -original project. - -With .NET Core 3.0, Microsoft shipped new https://devblogs.microsoft.com/dotnet/try-the-new-system-text-json-apis[System.Text.Json APIs] -that are included in-the-box with current versions of .NET. We have adopted -`System.Text.Json` for all serialization. Consumers can still define and register -their own `Serializer` implementation for their document types should they prefer -to use a different serialization library. - -By adopting `System.Text.Json`, we now depend on a well-maintained and supported -library from Microsoft. `System.Text.Json` is designed from the ground up to support -the latest performance optimizations in .NET and, as a result, provides both fast and low-allocation serialization. - -[discrete] -=== Mockability of ElasticsearchClient - -Testing code is an important part of software development. We recommend -that consumers prefer introducing an abstraction for their use of the {net-client} -as the prefered way to decouple consuming code from client types and support unit -testing. - -To support user testing scenarios, we have unsealed the `ElasticsearchClient` -type and made its methods virtual. This supports mocking the type directly for unit -testing. This is an improvement over the original `IElasticClient` interface from -`NEST` (v7) which only supported mocking of top-level client methods. - -We have also introduced a `TestableResponseFactory` in `Elastic.Transport` to -make it easier to create response instances with specific status codes and validity -that can be used during unit testing. - -These changes are in addition to our existing support for testing with an -`InMemoryConnection`, virtualized clusters and with our -https://github.com/elastic/elasticsearch-net-abstractions/blob/master/src/Elastic.Elasticsearch.Managed[`Elastic.Elasticsearch.Managed`] library for integration -testing against real {es} instances. - -[discrete] -=== Migrating to Elastic.Clients.Elasticsearch - -[WARNING] --- -The version 8 client does not currently have full-feature parity with `NEST`. The -client primary use case is for application developers communicating with {es}. --- - -The version 8 client focuses on core endpoints, more specifically for common CRUD -scenarios. The intention is to reduce the feature gap in subsequent versions. Review this documentation carefully to learn about the missing features and reduced API surface details before migrating from the v7 `NEST` client! - -The choice to code-generate a new evolution of the {net-client} introduces some -significant breaking changes. - -The v8 client is shipped as a new https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/[NuGet package] -which can be installed alongside v7 `NEST`. Some consumers may prefer a phased migration with both -packages side-by-side for a short period of time to manage complex migrations. In addition, `NEST` 7.17.x can continue to be used in -https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html[compatibility mode] -with {es} 8.x servers until the v8 {net-client} features -align with application requirements. - -[discrete] -=== Breaking Changes - -[WARNING] --- -As a result of code-generating a majority of the client types, version 8 of -the client includes multiple breaking changes. --- - -We have strived to keep the core foundation reasonably similar, but types emitted -through code-generation are subject to change between `NEST` (v7) and the new -`Elastic.Clients.Elasticsearch` (v8) package. - -[discrete] -==== Namespaces - -The package and top-level namespace for the v8 client have been renamed to -`Elastic.Clients.Elasticsearch`. All types belong to this namespace. When -necessary, to avoid potential conflicts, types are generated into suitable -sub-namespaces based on the https://github.com/elastic/elasticsearch-specification[{es} specification]. Additional `using` directives may be required to access such types -when using the {net-client}. - -Transport layer concepts have moved to the new `Elastic.Transport` NuGet package -and related types are defined under its namespace. Some configuration and low-level transport functionality may require a `using` directive for the `Elastic.Transport` -namespace. - -[discrete] -==== Type names - -Type names may have changed from previous versions. These are not listed explicitly due to the potentially vast number of subtle differences. -Type names will now more closely align to those used in the JSON and as documented -in the {es} documentation. - -[discrete] -==== Class members - -Types may include renamed properties based on the {es} specification, -which differ from the original `NEST` property names. The types used for properties -may also have changed due to code-generation. If you identify missing or -incorrectly-typed properties, please open https://github.com/elastic/elasticsearch-net/issues/new/choose[an issue] to alert us. - -[discrete] -==== Sealing classes - -Opinions on "sealing by default" within the .NET ecosystem tend to be quite -polarized. Microsoft seal all internal types for potential performance gains -and we see a benefit in starting with that approach for the {net-client}, -even for our public API surface. - -While it prevents inheritance and, therefore, may inhibit a few consumer scenarios, -sealing by default is intended to avoid the unexpected or invalid -extension of types that could inadvertently be broken in the future. - -[discrete] -==== Removed features - -As part of the clean-slate redesign of the new client, -certain features are removed from the v8.0 client. These are listed below: - -[discrete] -===== Attribute mappings - -In previous versions of the `NEST` client, attributes could be used to configure -the mapping behaviour and inference for user types. It is recommended that -mapping be completed via the fluent API when configuring client instances. -`System.Text.Json` attributes may be used to rename -and ignore properties during source serialization. - -[discrete] -===== CAT APIs - -The https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html[CAT APIs] -of {es} are intended for human-readable usage and will no longer be supported -via the v8 {net-client}. - -[discrete] -===== Interface removal - -Several interfaces are removed to simplify the library and avoid interfaces where only a -single implementation of that interface is expected to exist, such as -`IElasticClient` in `NEST`. Abstract base classes are preferred -over interfaces across the library, as this makes it easier to add enhancements -without introducing breaking changes for derived types. - -[discrete] -==== Missing features - -The following are some of the main features which -have not been re-implemented for the v8 client. -These might be reviewed and prioritized for inclusion in -future releases. - -* Query DSL operators for combining queries. -* Scroll Helper. -* Fluent API for union types. -* `AutoMap` for field datatype inference. -* Visitor pattern support for types such as `Properties`. -* Support for `JoinField` which affects `ChildrenAggregation`. -* Conditionless queries. -* DiagnosticSources have been removed in `Elastic.Transport` to provide a clean-slate -for an improved diagnostics story. The {net-client} emits https://opentelemetry.io/[OpenTelemetry] compatible `Activity` spans which can be consumed by APM agents such as the https://www.elastic.co/guide/en/apm/agent/dotnet/current/index.html[Elastic APM Agent for .NET]. -* Documentation is a work in progress, and we will expand on the documented scenarios -in future releases. - -[discrete] -=== Reduced API surface - -In the current versions of the code-generated .NET client, supporting commonly used -endpoints is critical. Some specific queries and aggregations need further work to generate code correctly, -hence they are not included yet. -Ensure that the features you are using are currently supported before migrating. - -An up to date list of all supported and unsupported endpoints can be found on https://github.com/elastic/elasticsearch-net/issues/7890[GitHub]. - -[discrete] -=== Workarounds for missing features - -If you encounter a missing feature with the v8 client, there are several ways to temporarily work around this issue until we officially reintroduce the feature. - -`NEST` 7.17.x can continue to be used in -https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html[compatibility mode] -with {es} 8.x servers until the v8 {net-client} features -align with application requirements. - -As a last resort, the low-level client `Elastic.Transport` can be used to create any desired request by hand: - -[source,csharp] ----- -public class MyRequestParameters : RequestParameters -{ - public bool Pretty - { - get => Q("pretty"); - init => Q("pretty", value); - } -} - -// ... - -var body = """ - { - "name": "my-api-key", - "expiration": "1d", - "...": "..." - } - """; - -MyRequestParameters requestParameters = new() -{ - Pretty = true -}; - -var pathAndQuery = requestParameters.CreatePathWithQueryStrings("/_security/api_key", - client.ElasticsearchClientSettings); -var endpointPath = new EndpointPath(Elastic.Transport.HttpMethod.POST, pathAndQuery); - -// Or, if the path does not contain query parameters: -// new EndpointPath(Elastic.Transport.HttpMethod.POST, "my_path") - -var response = await client.Transport - .RequestAsync( - endpointPath, - PostData.String(body), - null, - null, - cancellationToken: default) - .ConfigureAwait(false); ----- \ No newline at end of file diff --git a/docs/redirects.asciidoc b/docs/redirects.asciidoc deleted file mode 100644 index cb97e146ff1..00000000000 --- a/docs/redirects.asciidoc +++ /dev/null @@ -1,24 +0,0 @@ -["appendix",role="exclude",id="redirects"] -= Deleted pages - -The following pages have moved or been deleted. - -[role="exclude",id="configuration-options"] -== Configuration options - -This page has moved. See <>. - -[role="exclude",id="nest"] -== NEST - High level client - -This page has been deleted. - -[role="exclude",id="indexing-documents"] -== Indexing documents - -This page has been deleted. - -[role="exclude",id="bulkall-observable"] -== Multiple documents with `BulkAllObservable` helper - -This page has been deleted. \ No newline at end of file diff --git a/docs/reference/_options_on_elasticsearchclientsettings.md b/docs/reference/_options_on_elasticsearchclientsettings.md new file mode 100644 index 00000000000..49d5f904bd6 --- /dev/null +++ b/docs/reference/_options_on_elasticsearchclientsettings.md @@ -0,0 +1,175 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/_options_on_elasticsearchclientsettings.html +--- + +# Options on ElasticsearchClientSettings [_options_on_elasticsearchclientsettings] + +The following is a list of available connection configuration options on `ElasticsearchClientSettings`: + +`Authentication` +: An implementation of `IAuthenticationHeader` describing what http header to use to authenticate with the product. + + ``` + `BasicAuthentication` for basic authentication + ``` + ``` + `ApiKey` for simple secret token + ``` + ``` + `Base64ApiKey` for Elastic Cloud style encoded api keys + ``` + + +`ClientCertificate` +: Use the following certificates to authenticate all HTTP requests. You can also set them on individual request using `ClientCertificates`. + +`ClientCertificates` +: Use the following certificates to authenticate all HTTP requests. You can also set them on individual request using `ClientCertificates`. + +`ConnectionLimit` +: Limits the number of concurrent connections that can be opened to an endpoint. Defaults to 80 (see `DefaultConnectionLimit`). + + For Desktop CLR, this setting applies to the `DefaultConnectionLimit` property on the `ServicePointManager` object when creating `ServicePoint` objects, affecting the default `IConnection` implementation. + + For Core CLR, this setting applies to the `MaxConnectionsPerServer` property on the `HttpClientHandler` instances used by the `HttpClient` inside the default `IConnection` implementation. + + +`DeadTimeout` +: The time to put dead nodes out of rotation (this will be multiplied by the number of times they’ve been dead). + +`DefaultDisableIdInference` +: Disables automatic Id inference for given CLR types. + + The client by default will use the value of a property named `Id` on a CLR type as the `_id` to send to {{es}}. Adding a type will disable this behaviour for that CLR type. If `Id` inference should be disabled for all CLR types, use `DefaultDisableIdInference`. + + +`DefaultFieldNameInferrer` +: Specifies how field names are inferred from CLR property names. + + By default, the client camel cases property names. For example, CLR property `EmailAddress` will be inferred as "emailAddress" {{es}} document field name. + + +`DefaultIndex` +: The default index to use for a request when no index has been explicitly specified and no default indices are specified for the given CLR type specified for the request. + +`DefaultMappingFor` +: Specify how the mapping is inferred for a given CLR type. The mapping can infer the index, id and relation name for a given CLR type, as well as control serialization behaviour for CLR properties. + +`DisableAutomaticProxyDetection` +: Disabled proxy detection on the webrequest, in some cases this may speed up the first connection your appdomain makes, in other cases it will actually increase the time for the first connection. No silver bullet! Use with care! + +`DisableDirectStreaming` +: When set to true will disable (de)serializing directly to the request and response stream and return a byte[] copy of the raw request and response. Defaults to false. + +`DisablePing` +: This signals that we do not want to send initial pings to unknown/previously dead nodes and just send the call straightaway. + +`DnsRefreshTimeout` +: DnsRefreshTimeout for the connections. Defaults to 5 minutes. + +`EnableDebugMode` +: Turns on settings that aid in debugging like `DisableDirectStreaming()` and `PrettyJson()` so that the original request and response JSON can be inspected. It also always asks the server for the full stack trace on errors. + +`EnableHttpCompression` +: Enable gzip compressed requests and responses. + +`EnableHttpPipelining` +: Whether HTTP pipelining is enabled. The default is `true`. + +`EnableTcpKeepAlive` +: Sets the keep-alive option on a TCP connection. + + For Desktop CLR, sets `ServicePointManager`.`SetTcpKeepAlive`. + + +`EnableTcpStats` +: Enable statistics about TCP connections to be collected when making a request. + +`GlobalHeaders` +: Try to send these headers for every request. + +`GlobalQueryStringParameters` +: Append these query string parameters automatically to every request. + +`MaxDeadTimeout` +: The maximum amount of time a node is allowed to marked dead. + +`MaximumRetries` +: When a retryable exception occurs or status code is returned this controls the maximum amount of times we should retry the call to {{es}}. + +`MaxRetryTimeout` +: Limits the total runtime including retries separately from `RequestTimeout`. When not specified defaults to `RequestTimeout` which itself defaults to 60 seconds. + +`MemoryStreamFactory` +: Provides a memory stream factory. + +`NodePredicate` +: Register a predicate to select which nodes that you want to execute API calls on. Note that sniffing requests omit this predicate and always execute on all nodes. When using an `IConnectionPool` implementation that supports reseeding of nodes, this will default to omitting master only node from regular API calls. When using static or single node connection pooling it is assumed the list of node you instantiate the client with should be taken verbatim. + +`OnRequestCompleted` +: Allows you to register a callback every time a an API call is returned. + +`OnRequestDataCreated` +: An action to run when the `RequestData` for a request has been created. + +`PingTimeout` +: The timeout in milliseconds to use for ping requests, which are issued to determine whether a node is alive. + +`PrettyJson` +: Provide hints to serializer and products to produce pretty, non minified json. + + Note: this is not a guarantee you will always get prettified json. + + +`Proxy` +: If your connection has to go through proxy, use this method to specify the proxy url. + +`RequestTimeout` +: The timeout in milliseconds for each request to {{es}}. + +`ServerCertificateValidationCallback` +: Register a `ServerCertificateValidationCallback` per request. + +`SkipDeserializationForStatusCodes` +: Configure the client to skip deserialization of certain status codes, for example, you run {{es}} behind a proxy that returns an unexpected json format. + +`SniffLifeSpan` +: Force a new sniff for the cluster when the cluster state information is older than the specified timespan. + +`SniffOnConnectionFault` +: Force a new sniff for the cluster state every time a connection dies. + +`SniffOnStartup` +: Sniff the cluster state immediately on startup. + +`ThrowExceptions` +: Instead of following a c/go like error checking on response. `IsValid` do throw an exception (except when `SuccessOrKnownError` is false) on the client when a call resulted in an exception on either the client or the {{es}} server. + + Reasons for such exceptions could be search parser errors, index missing exceptions, and so on. + + +`TransferEncodingChunked` +: Whether the request should be sent with chunked Transfer-Encoding. + +`UserAgent` +: The user agent string to send with requests. Useful for debugging purposes to understand client and framework versions that initiate requests to {{es}}. + +## ElasticsearchClientSettings with ElasticsearchClient [_elasticsearchclientsettings_with_elasticsearchclient] + +Here’s an example to demonstrate setting configuration options using the client. + +```csharp +var settings= new ElasticsearchClientSettings() + .DefaultMappingFor(i => i + .IndexName("my-projects") + .IdProperty(p => p.Name) + ) + .EnableDebugMode() + .PrettyJson() + .RequestTimeout(TimeSpan.FromMinutes(2)); + +var client = new ElasticsearchClient(settings); +``` + + diff --git a/docs/usage/aggregations.asciidoc b/docs/reference/aggregations.md similarity index 76% rename from docs/usage/aggregations.asciidoc rename to docs/reference/aggregations.md index 1f159763aaa..8a3ca8e8327 100644 --- a/docs/usage/aggregations.asciidoc +++ b/docs/reference/aggregations.md @@ -1,16 +1,19 @@ -[[aggregations]] -== Aggregation examples +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/aggregations.html +--- + +# Aggregation examples [aggregations] This page demonstrates how to use aggregations. -[discrete] -=== Top-level aggreggation -[discrete] -==== Fluent API +## Top-level aggreggation [_top_level_aggreggation] + + +### Fluent API [_fluent_api] -[source,csharp] ----- +```csharp var response = await client .SearchAsync(search => search .Index("persons") @@ -26,13 +29,12 @@ var response = await client ) .Size(10) ); ----- +``` -[discrete] -==== Object initializer API -[source,csharp] ----- +### Object initializer API [_object_initializer_api] + +```csharp var response = await client.SearchAsync(new SearchRequest("persons") { Query = Query.MatchAll(new MatchAllQuery()), @@ -45,25 +47,23 @@ var response = await client.SearchAsync(new SearchRequest("persons") }, Size = 10 }); ----- +``` + -[discrete] -==== Consume the response +### Consume the response [_consume_the_response] -[source,csharp] ----- +```csharp var max = response.Aggregations!.GetMax("agg_name")!; Console.WriteLine(max.Value); ----- +``` -[discrete] -=== Sub-aggregation -[discrete] -==== Fluent API +## Sub-aggregation [_sub_aggregation] -[source,csharp] ----- + +### Fluent API [_fluent_api_2] + +```csharp var response = await client .SearchAsync(search => search .Index("persons") @@ -86,13 +86,12 @@ var response = await client ) .Size(10) ); ----- +``` + -[discrete] -==== Object initializer API +### Object initializer API [_object_initializer_api_2] -[source,csharp] ----- +```csharp var topLevelAggregation = Aggregation.Terms(new TermsAggregation { Field = Infer.Field(x => x.FirstName) @@ -115,17 +114,17 @@ var response = await client.SearchAsync(new SearchRequest("persons") }, Size = 10 }); ----- +``` -[discrete] -==== Consume the response -[source,csharp] ----- +### Consume the response [_consume_the_response_2] + +```csharp var firstnames = response.Aggregations!.GetStringTerms("firstnames")!; foreach (var bucket in firstnames.Buckets) { var avg = bucket.Aggregations.GetAverage("avg_age")!; Console.WriteLine($"The average age for persons named '{bucket.Key}' is {avg}"); } ----- +``` + diff --git a/docs/reference/client-concepts.md b/docs/reference/client-concepts.md new file mode 100644 index 00000000000..73655604f4a --- /dev/null +++ b/docs/reference/client-concepts.md @@ -0,0 +1,9 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/client-concepts.html +--- + +# Client concepts [client-concepts] + +The .NET client for {{es}} maps closely to the original {{es}} API. All +requests and responses are exposed through types, making it ideal for getting up and running quickly. diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md new file mode 100644 index 00000000000..0577876926e --- /dev/null +++ b/docs/reference/configuration.md @@ -0,0 +1,10 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/configuration.html +--- + +# Configuration [configuration] + +Connecting to {{es}} with the client is easy, but it’s possible that you’d like to change the default connection behaviour. There are a number of configuration options available on `ElasticsearchClientSettings` that can be used to control how the client interact with {{es}}. + + diff --git a/docs/reference/connecting.md b/docs/reference/connecting.md new file mode 100644 index 00000000000..5c7fc326ca2 --- /dev/null +++ b/docs/reference/connecting.md @@ -0,0 +1,123 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/connecting.html +--- + +# Connecting [connecting] + +This page contains the information you need to create an instance of the .NET Client for {{es}} that connects to your {{es}} cluster. + +It’s possible to connect to your {{es}} cluster via a single node, or by specifying multiple nodes using a node pool. Using a node pool has a few advantages over a single node, such as load balancing and cluster failover support. The client provides convenient configuration options to connect to an Elastic Cloud deployment. + +::::{important} +Client applications should create a single instance of `ElasticsearchClient` that is used throughout your application for its entire lifetime. Internally the client manages and maintains HTTP connections to nodes, reusing them to optimize performance. If you use a dependency injection container for your application, the client instance should be registered with a singleton lifetime. +:::: + + + +## Connecting to a cloud deployment [cloud-deployment] + +[Elastic Cloud](docs-content://deploy-manage/deploy/elastic-cloud/cloud-hosted.md) is the easiest way to get started with {{es}}. When connecting to Elastic Cloud with the .NET {{es}} client you should always use the Cloud ID. You can find this value within the "Manage Deployment" page after you’ve created a cluster (look in the top-left if you’re in Kibana). + +We recommend using a Cloud ID whenever possible because your client will be automatically configured for optimal use with Elastic Cloud, including HTTPS and HTTP compression. + +Connecting to an Elasticsearch Service deployment is achieved by providing the unique Cloud ID for your deployment when configuring the `ElasticsearchClient` instance. You also require suitable credentials, either a username and password or an API key that your application uses to authenticate with your deployment. + +As a security best practice, it is recommended to create a dedicated API key per application, with permissions limited to only those required for any API calls the application is authorized to make. + +The following snippet demonstrates how to create a client instance that connects to an {{es}} deployment in the cloud. + +```csharp +using Elastic.Clients.Elasticsearch; +using Elastic.Transport; + +var client = new ElasticsearchClient("", new ApiKey("")); <1> +``` + +1. Replace the placeholder string values above with your cloud ID and the API key configured for your application to access your deployment. + + + +## Connecting to a single node [single-node] + +Single node configuration is best suited to connections to a multi-node cluster running behind a load balancer or reverse proxy, which is exposed via a single URL. It may also be convenient to use a single node during local application development. If the URL represents a single {{es}} node, be aware that this offers no resiliency should the server be unreachable or unresponsive. + +By default, security features such as authentication and TLS are enabled on {{es}} clusters. When you start {{es}} for the first time, TLS is configured automatically for the HTTP layer. A CA certificate is generated and stored on disk which is used to sign the certificates for the HTTP layer of the {{es}} cluster. + +In order for the client to establish a connection with the cluster over HTTPS, the CA certificate must be trusted by the client application. The simplest choice is to use the hex-encoded SHA-256 fingerprint of the CA certificate. The CA fingerprint is output to the terminal when you start {{es}} for the first time. You’ll see a distinct block like the one below in the output from {{es}} (you may have to scroll up if it’s been a while): + +```sh +---------------------------------------------------------------- +-> Elasticsearch security features have been automatically configured! +-> Authentication is enabled and cluster connections are encrypted. + +-> Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`): + lhQpLELkjkrawaBoaz0Q + +-> HTTP CA certificate SHA-256 fingerprint: + a52dd93511e8c6045e21f16654b77c9ee0f34aea26d9f40320b531c474676228 +... +---------------------------------------------------------------- +``` + +Note down the `elastic` user password and HTTP CA fingerprint for the next sections. + +The CA fingerprint can also be retrieved at any time from a running cluster using the following command: + +```shell +openssl x509 -fingerprint -sha256 -in config/certs/http_ca.crt +``` + +The command returns the security certificate, including the fingerprint. The `issuer` should be `Elasticsearch security auto-configuration HTTP CA`. + +```shell +issuer= /CN=Elasticsearch security auto-configuration HTTP CA +SHA256 Fingerprint= +``` + +Visit the [Start the Elastic Stack with security enabled automatically](docs-content://deploy-manage/deploy/self-managed/installing-elasticsearch.md) documentation for more information. + +The following snippet shows you how to create a client instance that connects to your {{es}} cluster via a single node, using the CA fingerprint: + +```csharp +using Elastic.Clients.Elasticsearch; +using Elastic.Transport; + +var settings = new ElasticsearchClientSettings(new Uri("https://localhost:9200")) + .CertificateFingerprint("") + .Authentication(new BasicAuthentication("", "")); + +var client = new ElasticsearchClient(settings); +``` + +The preceding snippet demonstrates configuring the client to authenticate by providing a username and password with basic authentication. If preferred, you may also use `ApiKey` authentication as shown in the cloud connection example. + + +## Connecting to multiple nodes using a node pool [multiple-nodes] + +To provide resiliency, you should configure multiple nodes for your cluster to which the client attempts to communicate. By default, the client cycles through nodes for each request in a round robin fashion. The client also tracks unhealthy nodes and avoids sending requests to them until they become healthy. + +This configuration is best suited to connect to a known small sized cluster, where you do not require sniffing to detect the cluster topology. + +The following snippet shows you how to connect to multiple nodes by using a static node pool: + +```csharp +using Elastic.Clients.Elasticsearch; +using Elastic.Transport; + +var nodes = new Uri[] +{ + new Uri("https://myserver1:9200"), + new Uri("https://myserver2:9200"), + new Uri("https://myserver3:9200") +}; + +var pool = new StaticNodePool(nodes); + +var settings = new ElasticsearchClientSettings(pool) + .CertificateFingerprint("") + .Authentication(new ApiKey("")); + +var client = new ElasticsearchClient(settings); +``` + diff --git a/docs/reference/esql.md b/docs/reference/esql.md new file mode 100644 index 00000000000..f1db14c16cc --- /dev/null +++ b/docs/reference/esql.md @@ -0,0 +1,50 @@ +--- +navigation_title: "Using ES|QL" +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/esql.html +--- + +# ES|QL in the .NET client [esql] + + +This page helps you understand and use [ES|QL](docs-content://explore-analyze/query-filter/languages/esql.md) in the .NET client. + +There are two ways to use ES|QL in the .NET client: + +* Use the Elasticsearch [ES|QL API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-esql) directly: This is the most flexible approach, but it’s also the most complex because you must handle results in their raw form. You can choose the precise format of results, such as JSON, CSV, or text. +* Use ES|QL high-level helpers: These helpers take care of parsing the raw response into something readily usable by the application. Several helpers are available for different use cases, such as object mapping, cursor traversal of results (in development), and dataframes (in development). + + +## How to use the ES|QL API [esql-how-to] + +The [ES|QL query API](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-esql) allows you to specify how results should be returned. You can choose a [response format](docs-content://explore-analyze/query-filter/languages/esql-rest.md#esql-rest-format) such as CSV, text, or JSON, then fine-tune it with parameters like column separators and locale. + +The following example gets ES|QL results as CSV and parses them: + +```csharp +var response = await client.Esql.QueryAsync(r => r + .Query("FROM index") + .Format("csv") +); +var csvContents = Encoding.UTF8.GetString(response.Data); +``` + + +## Consume ES|QL results [esql-consume-results] + +The previous example showed that although the raw ES|QL API offers maximum flexibility, additional work is required in order to make use of the result data. + +To simplify things, try working with these three main representations of ES|QL results (each with its own mapping helper): + +* **Objects**, where each row in the results is mapped to an object from your application domain. This is similar to what ORMs (object relational mappers) commonly do. +* **Cursors**, where you scan the results row by row and access the data using column names. This is similar to database access libraries. +* **Dataframes**, where results are organized in a column-oriented structure that allows efficient processing of column data. + +```csharp +// ObjectAPI example +var response = await client.Esql.QueryAsObjectsAsync(x => x.Query("FROM index")); +foreach (var person in response) +{ + // ... +} +``` diff --git a/docs/reference/examples.md b/docs/reference/examples.md new file mode 100644 index 00000000000..67cc8034ae0 --- /dev/null +++ b/docs/reference/examples.md @@ -0,0 +1,164 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/examples.html +--- + +# CRUD usage examples [examples] + +This page helps you to understand how to perform various basic {{es}} CRUD (create, read, update, delete) operations using the .NET client. It demonstrates how to create a document by indexing an object into {{es}}, read a document back, retrieving it by ID or performing a search, update one of the fields in a document and delete a specific document. + +These examples assume you have an instance of the `ElasticsearchClient` accessible via a local variable named `client` and several using directives in your C# file. + +```csharp +using System; +using Elastic.Clients.Elasticsearch; +using Elastic.Clients.Elasticsearch.QueryDsl; +var client = new ElasticsearchClient(); <1> +``` + +1. The default constructor, assumes an unsecured {{es}} server is running and exposed on *http://localhost:9200*. See [connecting](/reference/connecting.md) for examples of connecting to secured servers and [Elastic Cloud](https://www.elastic.co/cloud) deployments. + + +The examples operate on data representing tweets. Tweets are modelled in the client application using a C# class named *Tweet* containing several properties that map to the document structure being stored in {{es}}. + +```csharp +public class Tweet +{ + public int Id { get; set; } <1> + public string User { get; set; } + public DateTime PostDate { get; set; } + public string Message { get; set; } +} +``` + +1. By default, the .NET client will try to find a property called `Id` on the class. When such a property is present it will index the document into {{es}} using the ID specified by the value of this property. + + + +## Indexing a document [indexing-net] + +Documents can be indexed by creating an instance representing a tweet and indexing it via the client. In these examples, we will work with an index named *my-tweet-index*. + +```csharp +var tweet = new Tweet <1> +{ + Id = 1, + User = "stevejgordon", + PostDate = new DateTime(2009, 11, 15), + Message = "Trying out the client, so far so good?" +}; + +var response = await client.IndexAsync(tweet, "my-tweet-index"); <2> + +if (response.IsValidResponse) <3> +{ + Console.WriteLine($"Index document with ID {response.Id} succeeded."); <4> +} +``` + +1. Create an instance of the `Tweet` class with relevant properties set. +2. Prefer the async APIs, which require awaiting the response. +3. Check the `IsValid` property on the response to confirm that the request and operation succeeded. +4. Access the `IndexResponse` properties, such as the ID, if necessary. + + + +## Getting a document [getting-net] + +```csharp +var response = await client.GetAsync(1, idx => idx.Index("my-tweet-index")); <1> + +if (response.IsValidResponse) +{ + var tweet = response.Source; <2> +} +``` + +1. The `GetResponse` is mapped 1-to-1 with the Elasticsearch JSON response. +2. The original document is deserialized as an instance of the Tweet class, accessible on the response via the `Source` property. + + + +## Searching for documents [searching-net] + +The client exposes a fluent interface and a powerful query DSL for searching. + +```csharp +var response = await client.SearchAsync(s => s <1> + .Index("my-tweet-index") <2> + .From(0) + .Size(10) + .Query(q => q + .Term(t => t.User, "stevejgordon") <3> + ) +); + +if (response.IsValidResponse) +{ + var tweet = response.Documents.FirstOrDefault(); <4> +} +``` + +1. The generic type argument specifies the `Tweet` class, which is used when deserialising the hits from the response. +2. The index can be omitted if a `DefaultIndex` has been configured on `ElasticsearchClientSettings`, or a specific index was configured when mapping this type. +3. Execute a term query against the `user` field, searching for tweets authored by the user *stevejgordon*. +4. Documents matched by the query are accessible via the `Documents` collection property on the `SearchResponse`. + + +You may prefer using the object initializer syntax for requests if lambdas aren’t your thing. + +```csharp +var request = new SearchRequest("my-tweet-index") <1> +{ + From = 0, + Size = 10, + Query = new TermQuery("user") { Value = "stevejgordon" } +}; + +var response = await client.SearchAsync(request); <2> + +if (response.IsValidResponse) +{ + var tweet = response.Documents.FirstOrDefault(); +} +``` + +1. Create an instance of `SearchRequest`, setting properties to control the search operation. +2. Pass the request to the `SearchAsync` method on the client. + + + +## Updating documents [updating-net] + +Documents can be updated in several ways, including by providing a complete replacement for an existing document ID. + +```csharp +tweet.Message = "This is a new message"; <1> + +var response = await client.UpdateAsync("my-tweet-index", 1, u => u + .Doc(tweet)); <2> + +if (response.IsValidResponse) +{ + Console.WriteLine("Update document succeeded."); +} +``` + +1. Update a property on the existing tweet instance. +2. Send the updated tweet object in the update request. + + + +## Deleting documents [deleting-net] + +Documents can be deleted by providing the ID of the document to remove. + +```csharp +var response = await client.DeleteAsync("my-tweet-index", 1); + +if (response.IsValidResponse) +{ + Console.WriteLine("Delete document succeeded."); +} +``` + diff --git a/docs/reference/getting-started.md b/docs/reference/getting-started.md new file mode 100644 index 00000000000..48ac2a872d9 --- /dev/null +++ b/docs/reference/getting-started.md @@ -0,0 +1,144 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/getting-started-net.html + - https://www.elastic.co/guide/en/serverless/current/elasticsearch-dot-net-client-getting-started.html +--- + +# Getting started [getting-started-net] + +This page guides you through the installation process of the .NET client, shows you how to instantiate the client, and how to perform basic Elasticsearch operations with it. + + +### Requirements [_requirements] + +* .NET Core, .NET 5+ or .NET Framework (4.6.1 and higher). + + +### Installation [_installation] + +To install the latest version of the client for SDK style projects, run the following command: + +```shell +dotnet add package Elastic.Clients.Elasticsearch +``` + +Refer to the [*Installation*](/reference/installation.md) page to learn more. + + +### Connecting [_connecting] + +You can connect to the Elastic Cloud using an API key and the Elasticsearch endpoint. + +```csharp +var client = new ElasticsearchClient("", new ApiKey("")); +``` + +Your Elasticsearch endpoint can be found on the **My deployment** page of your deployment: + +:::{image} ../images/es-endpoint.jpg +:alt: Finding Elasticsearch endpoint +::: + +You can generate an API key on the **Management** page under Security. + +:::{image} ../images/create-api-key.png +:alt: Create API key +::: + +For other connection options, refer to the [*Connecting*](/reference/connecting.md) section. + + +### Operations [_operations] + +Time to use Elasticsearch! This section walks you through the basic, and most important, operations of Elasticsearch. For more operations and more advanced examples, refer to the [*CRUD usage examples*](/reference/examples.md) page. + + +#### Creating an index [_creating_an_index] + +This is how you create the `my_index` index: + +```csharp +var response = await client.Indices.CreateAsync("my_index"); +``` + + +#### Indexing documents [_indexing_documents] + +This is a simple way of indexing a document: + +```csharp +var doc = new MyDoc +{ + Id = 1, + User = "flobernd", + Message = "Trying out the client, so far so good?" +}; + +var response = await client.IndexAsync(doc, "my_index"); +``` + + +#### Getting documents [_getting_documents] + +You can get documents by using the following code: + +```csharp +var response = await client.GetAsync(id, idx => idx.Index("my_index")); + +if (response.IsValidResponse) +{ + var doc = response.Source; +} +``` + + +#### Searching documents [_searching_documents] + +This is how you can create a single match query with the .NET client: + +```csharp +var response = await client.SearchAsync(s => s + .Index("my_index") + .From(0) + .Size(10) + .Query(q => q + .Term(t => t.User, "flobernd") + ) +); + +if (response.IsValidResponse) +{ + var doc = response.Documents.FirstOrDefault(); +} +``` + + +#### Updating documents [_updating_documents] + +This is how you can update a document, for example to add a new field: + +```csharp +doc.Message = "This is a new message"; + +var response = await client.UpdateAsync("my_index", 1, u => u + .Doc(doc)); +``` + + +#### Deleting documents [_deleting_documents] + +```csharp +var response = await client.DeleteAsync("my_index", 1); +``` + + +#### Deleting an index [_deleting_an_index] + +```csharp +var response = await client.Indices.DeleteAsync("my_index"); +``` + + +## Further reading [_further_reading] + +* Refer to the [*Usage recommendations*](/reference/recommendations.md) page to learn more about how to use the client the most efficiently. diff --git a/docs/reference/index.md b/docs/reference/index.md new file mode 100644 index 00000000000..90ef20d616f --- /dev/null +++ b/docs/reference/index.md @@ -0,0 +1,44 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/index.html + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/introduction.html +--- + +# .NET [introduction] + +**Rapidly develop applications with the .NET client for {{es}}.** + +Designed for .NET application developers, the .NET language client library provides a strongly typed API and query DSL for interacting with {{es}}. The .NET client includes higher-level abstractions, such as helpers for coordinating bulk indexing and update operations. It also comes with built-in, configurable cluster failover retry mechanisms. + +The {{es}} .NET client is available as a [NuGet](https://www.nuget.org/packages/Elastic.Clients.Elasticsearch) package for use with .NET Core, .NET 5+, and .NET Framework (4.6.1 and later) applications. + +*NOTE: This documentation covers the v8 .NET client for {{es}}, for use with {{es}} 8.x versions. To develop applications targeting {{es}} v7, use the [v7 (NEST) client](https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17).* + + +## Features [features] + +* One-to-one mapping with the REST API. +* Strongly typed requests and responses for {{es}} APIs. +* Fluent API for building requests. +* Query DSL to assist with constructing search queries. +* Helpers for common tasks such as bulk indexing of documents. +* Pluggable serialization of requests and responses based on `System.Text.Json`. +* Diagnostics, auditing, and .NET activity integration. + +The .NET {{es}} client is built on the Elastic Transport library, which provides: + +* Connection management and load balancing across all available nodes. +* Request retries and dead connections handling. + + +## {{es}} version compatibility [_es_version_compatibility] + +Language clients are forward compatible: clients support communicating with current and later minor versions of {{es}}. {{es}} language clients are backward compatible with default distributions only and without guarantees. + + +## Questions, bugs, comments, feature requests [_questions_bugs_comments_feature_requests] + +To submit a bug report or feature request, use [GitHub issues](https://github.com/elastic/elasticsearch-net/issues). + +For more general questions and comments, try the community forum on [discuss.elastic.co](https://discuss.elastic.co/c/elasticsearch). Mention `.NET` in the title to indicate the discussion topic. + diff --git a/docs/reference/installation.md b/docs/reference/installation.md new file mode 100644 index 00000000000..8fbb2dcd2e9 --- /dev/null +++ b/docs/reference/installation.md @@ -0,0 +1,67 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/installation.html +--- + +# Installation [installation] + +This page shows you how to install the .NET client for {{es}}. + +::::{important} +The v8 client for .NET does not have complete feature parity with the v7 `NEST` client. It may not be suitable for for all applications until additional endpoints and features are supported. We therefore recommend you thoroughly review our [release notes](/release-notes/index.md) before attempting to migrate existing applications to the `Elastic.Clients.Elasticsearch` library. Until the new client supports all endpoints and features your application requires, you may continue to use the 7.17.x [NEST](https://www.nuget.org/packages/NEST) client to communicate with v8 Elasticsearch servers using compatibility mode. Refer to the [Connecting to Elasticsearch v8.x using the v7.17.x client documentation](https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html) for guidance on configuring the 7.17.x client. +:::: + + + +## Installing the .NET client [dot-net-client] + +For SDK style projects, you can install the {{es}} client by running the following .NET CLI command in your terminal: + +```text +dotnet add package Elastic.Clients.Elasticsearch +``` + +This command adds a package reference to your project (csproj) file for the latest stable version of the client. + +If you prefer, you may also manually add a package reference inside your project file: + +```shell + +``` + +*NOTE: The version number should reflect the latest published version from [NuGet.org](https://www.nuget.org/packages/Elastic.Clients.Elasticsearch). To install a different version, modify the version as necessary.* + +For Visual Studio users, the .NET client can also be installed from the Package Manager Console inside Visual Studio using the following command: + +```shell +Install-Package Elastic.Clients.Elasticsearch +``` + +Alternatively, search for `Elastic.Clients.Elasticsearch` in the NuGet Package Manager UI. + +To learn how to connect the {{es}} client, refer to the [Connecting](/reference/connecting.md) section. + + +## Compatibility [compatibility] + +The {{es}} client is compatible with currently maintained .NET runtime versions. Compatibility with End of Life (EOL) .NET runtimes is not guaranteed or supported. + +Language clients are forward compatible; meaning that the clients support communicating with greater or equal minor versions of {{es}} without breaking. It does not mean that the clients automatically support new features of newer {{es}} versions; it is only possible after a release of a new client version. For example, a 8.12 client version won’t automatically support the new features of the 8.13 version of {{es}}, the 8.13 client version is required for that. {{es}} language clients are only backwards compatible with default distributions and without guarantees made. + +| Elasticsearch Version | Elasticsearch-NET Branch | Supported | +| --- | --- | --- | +| main | main | | +| 8.x | 8.x | 8.x | +| 7.x | 7.x | 7.17 | + +Refer to the [end-of-life policy](https://www.elastic.co/support/eol) for more information. + + +## CI feed [ci-feed] + +We publish CI builds of our client packages, including the latest unreleased features. If you want to experiment with the latest bits, you can add the CI feed to your list of NuGet package sources. + +Feed URL: [https://f.feedz.io/elastic/all/nuget/index.json](https://f.feedz.io/elastic/all/nuget/index.json) + +We do not recommend using CI builds for production applications as they are not formally supported until they are released. + diff --git a/docs/usage/mappings.asciidoc b/docs/reference/mappings.md similarity index 60% rename from docs/usage/mappings.asciidoc rename to docs/reference/mappings.md index 13d62f63147..b28b8ba4a5e 100644 --- a/docs/usage/mappings.asciidoc +++ b/docs/reference/mappings.md @@ -1,13 +1,16 @@ -[[mappings]] -== Custom mapping examples +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/mappings.html +--- + +# Custom mapping examples [mappings] This page demonstrates how to configure custom mappings on an index. -[discrete] -=== Configure mappings during index creation -[source,csharp] ----- +## Configure mappings during index creation [_configure_mappings_during_index_creation] + +```csharp await client.Indices.CreateAsync(index => index .Index("index") .Mappings(mappings => mappings @@ -17,13 +20,12 @@ await client.Indices.CreateAsync(index => index ) ) ); ----- +``` -[discrete] -=== Configure mappings after index creation -[source,csharp] ----- +## Configure mappings after index creation [_configure_mappings_after_index_creation] + +```csharp await client.Indices.PutMappingAsync(mappings => mappings .Indices("index") .Properties(properties => properties @@ -31,4 +33,5 @@ await client.Indices.PutMappingAsync(mappings => mappings .Keyword(x => x.FirstName!, keyword => keyword.Index(false)) ) ); ----- +``` + diff --git a/docs/reference/migration-guide.md b/docs/reference/migration-guide.md new file mode 100644 index 00000000000..aed73bc3d95 --- /dev/null +++ b/docs/reference/migration-guide.md @@ -0,0 +1,222 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/migration-guide.html +--- + +# Migration guide: From NEST v7 to .NET Client v8 [migration-guide] + +The following migration guide explains the current state of the client, missing features, breaking changes and our rationale for some of the design choices we have introduced. + + +## Version 8 is a refresh [_version_8_is_a_refresh] + +::::{important} +It is important to highlight that v8 of the Elasticsearch .NET Client represents a new start for the client design. It is important to review how this may affect your code and usage. + +:::: + + +Mature code becomes increasingly hard to maintain over time. Major releases allow us to simplify and better align our language clients with each other in terms of design. It is crucial to find the right balance between uniformity across programming languages and the idiomatic concerns of each language. For .NET, we typically compare and contrast with [Java](https://github.com/elastic/elasticsearch-java) and [Go](https://github.com/elastic/go-elasticsearch) to make sure that our approach is equivalent for each of these. We also take heavy inspiration from Microsoft framework design guidelines and the conventions of the wider .NET community. + + +### New Elastic.Clients.Elasticsearch NuGet package [_new_elastic_clients_elasticsearch_nuget_package] + +We have shipped the new code-generated client as a [NuGet package](https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/) with a new root namespace, `Elastic.Clients.Elasticsearch`. The v8 client is built upon the foundations of the v7 `NEST` client, but there are changes. By shipping as a new package, the expectation is that migration can be managed with a phased approach. + +While this is a new package, we have aligned the major version (v8.x.x) with the supported {{es}} server version to clearly indicate the client/server compatibility. The v8 client is designed to work with version 8 of {{es}}. + +The v7 `NEST` client continues to be supported but will not gain new features or support for new {{es}} endpoints. It should be considered deprecated in favour of the new client. + + +### Limited feature set [_limited_feature_set] + +::::{warning} +The version 8 Elasticsearch .NET Client does not have feature parity with the previous v7 `NEST` high-level client. + +:::: + + +If a feature you depend on is missing (and not explicitly documented below as a feature that we do not plan to reintroduce), open [an issue](https://github.com/elastic/elasticsearch-net/issues/new/choose) or comment on a relevant existing issue to highlight your need to us. This will help us prioritise our roadmap. + + +## Code generation [_code_generation] + +Given the size of the {{es}} API surface today, it is no longer practical to maintain thousands of types (requests, responses, queries, aggregations, etc.) by hand. To ensure consistent, accurate, and timely alignment between language clients and {{es}}, the 8.x clients, and many of the associated types are now automatically code-generated from a [shared specification](https://github.com/elastic/elasticsearch-specification). This is a common solution to maintaining alignment between client and server among SDKs and libraries, such as those for Azure, AWS and the Google Cloud Platform. + +Code-generation from a specification has inevitably led to some differences between the existing v7 `NEST` types and those available in the new v7 Elasticsearch .NET Client. For version 8, we generate strictly from the specification, special casing a few areas to improve usability or to align with language idioms. + +The base type hierarchy for concepts such as `Properties`, `Aggregations` and `Queries` is no longer present in generated code, as these arbitrary groupings do not align with concrete concepts of the public server API. These considerations do not preclude adding syntactic sugar and usability enhancements to types in future releases on a case-by-case basis. + + +## Elastic.Transport [_elastic_transport] + +The .NET client includes a transport layer responsible for abstracting HTTP concepts and to provide functionality such as our request pipeline. This supports round-robin load-balancing of requests to nodes, pinging failed nodes and sniffing the cluster for node roles. + +In v7, this layer shipped as `Elasticsearch.Net` and was considered our low-level client which could be used to send and receive raw JSON bytes between the client and server. + +As part of the work for 8.0.0, we have moved the transport layer out into a [new dedicated package](https://www.nuget.org/packages/Elastic.Transport) and [repository](https://github.com/elastic/elastic-transport-net), named `Elastic.Transport`. This supports reuse across future clients and allows consumers with extremely high-performance requirements to build upon this foundation. + + +## System.Text.Json for serialization [_system_text_json_for_serialization] + +The v7 `NEST` high-level client used an internalized and modified version of [Utf8Json](https://github.com/neuecc/Utf8Json) for request and response serialization. This was introduced for its performance improvements over [Json.NET](https://www.newtonsoft.com/json), the more common JSON framework at the time. + +While Utf8Json provides good value, we have identified minor bugs and performance issues that have required maintenance over time. Some of these are hard to change without more significant effort. This library is no longer maintained, and any such changes cannot easily be contributed back to the original project. + +With .NET Core 3.0, Microsoft shipped new [System.Text.Json APIs](https://devblogs.microsoft.com/dotnet/try-the-new-system-text-json-apis) that are included in-the-box with current versions of .NET. We have adopted `System.Text.Json` for all serialization. Consumers can still define and register their own `Serializer` implementation for their document types should they prefer to use a different serialization library. + +By adopting `System.Text.Json`, we now depend on a well-maintained and supported library from Microsoft. `System.Text.Json` is designed from the ground up to support the latest performance optimizations in .NET and, as a result, provides both fast and low-allocation serialization. + + +## Mockability of ElasticsearchClient [_mockability_of_elasticsearchclient] + +Testing code is an important part of software development. We recommend that consumers prefer introducing an abstraction for their use of the Elasticsearch .NET Client as the prefered way to decouple consuming code from client types and support unit testing. + +To support user testing scenarios, we have unsealed the `ElasticsearchClient` type and made its methods virtual. This supports mocking the type directly for unit testing. This is an improvement over the original `IElasticClient` interface from `NEST` (v7) which only supported mocking of top-level client methods. + +We have also introduced a `TestableResponseFactory` in `Elastic.Transport` to make it easier to create response instances with specific status codes and validity that can be used during unit testing. + +These changes are in addition to our existing support for testing with an `InMemoryConnection`, virtualized clusters and with our [`Elastic.Elasticsearch.Managed`](https://github.com/elastic/elasticsearch-net-abstractions/blob/master/src/Elastic.Elasticsearch.Managed) library for integration testing against real {{es}} instances. + + +## Migrating to Elastic.Clients.Elasticsearch [_migrating_to_elastic_clients_elasticsearch] + +::::{warning} +The version 8 client does not currently have full-feature parity with `NEST`. The client primary use case is for application developers communicating with {{es}}. + +:::: + + +The version 8 client focuses on core endpoints, more specifically for common CRUD scenarios. The intention is to reduce the feature gap in subsequent versions. Review this documentation carefully to learn about the missing features and reduced API surface details before migrating from the v7 `NEST` client! + +The choice to code-generate a new evolution of the Elasticsearch .NET Client introduces some significant breaking changes. + +The v8 client is shipped as a new [NuGet package](https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/) which can be installed alongside v7 `NEST`. Some consumers may prefer a phased migration with both packages side-by-side for a short period of time to manage complex migrations. In addition, `NEST` 7.17.x can continue to be used in [compatibility mode](https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html) with {{es}} 8.x servers until the v8 Elasticsearch .NET Client features align with application requirements. + + +## Breaking Changes [_breaking_changes] + +::::{warning} +As a result of code-generating a majority of the client types, version 8 of the client includes multiple breaking changes. + +:::: + + +We have strived to keep the core foundation reasonably similar, but types emitted through code-generation are subject to change between `NEST` (v7) and the new `Elastic.Clients.Elasticsearch` (v8) package. + + +### Namespaces [_namespaces] + +The package and top-level namespace for the v8 client have been renamed to `Elastic.Clients.Elasticsearch`. All types belong to this namespace. When necessary, to avoid potential conflicts, types are generated into suitable sub-namespaces based on the [{{es}} specification](https://github.com/elastic/elasticsearch-specification). Additional `using` directives may be required to access such types when using the Elasticsearch .NET Client. + +Transport layer concepts have moved to the new `Elastic.Transport` NuGet package and related types are defined under its namespace. Some configuration and low-level transport functionality may require a `using` directive for the `Elastic.Transport` namespace. + + +### Type names [_type_names] + +Type names may have changed from previous versions. These are not listed explicitly due to the potentially vast number of subtle differences. Type names will now more closely align to those used in the JSON and as documented in the {{es}} documentation. + + +### Class members [_class_members] + +Types may include renamed properties based on the {{es}} specification, which differ from the original `NEST` property names. The types used for properties may also have changed due to code-generation. If you identify missing or incorrectly-typed properties, please open [an issue](https://github.com/elastic/elasticsearch-net/issues/new/choose) to alert us. + + +### Sealing classes [_sealing_classes] + +Opinions on "sealing by default" within the .NET ecosystem tend to be quite polarized. Microsoft seal all internal types for potential performance gains and we see a benefit in starting with that approach for the Elasticsearch .NET Client, even for our public API surface. + +While it prevents inheritance and, therefore, may inhibit a few consumer scenarios, sealing by default is intended to avoid the unexpected or invalid extension of types that could inadvertently be broken in the future. + + +### Removed features [_removed_features] + +As part of the clean-slate redesign of the new client, certain features are removed from the v8.0 client. These are listed below: + + +#### Attribute mappings [_attribute_mappings] + +In previous versions of the `NEST` client, attributes could be used to configure the mapping behaviour and inference for user types. It is recommended that mapping be completed via the fluent API when configuring client instances. `System.Text.Json` attributes may be used to rename and ignore properties during source serialization. + + +#### CAT APIs [_cat_apis] + +The [CAT APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-cat) of {{es}} are intended for human-readable usage and will no longer be supported via the v8 Elasticsearch .NET Client. + + +#### Interface removal [_interface_removal] + +Several interfaces are removed to simplify the library and avoid interfaces where only a single implementation of that interface is expected to exist, such as `IElasticClient` in `NEST`. Abstract base classes are preferred over interfaces across the library, as this makes it easier to add enhancements without introducing breaking changes for derived types. + + +### Missing features [_missing_features] + +The following are some of the main features which have not been re-implemented for the v8 client. These might be reviewed and prioritized for inclusion in future releases. + +* Query DSL operators for combining queries. +* Scroll Helper. +* Fluent API for union types. +* `AutoMap` for field datatype inference. +* Visitor pattern support for types such as `Properties`. +* Support for `JoinField` which affects `ChildrenAggregation`. +* Conditionless queries. +* DiagnosticSources have been removed in `Elastic.Transport` to provide a clean-slate for an improved diagnostics story. The Elasticsearch .NET Client emits [OpenTelemetry](https://opentelemetry.io/) compatible `Activity` spans which can be consumed by APM agents such as the [Elastic APM Agent for .NET](apm-agent-dotnet://docs/reference/index.md). +* Documentation is a work in progress, and we will expand on the documented scenarios in future releases. + + +## Reduced API surface [_reduced_api_surface] + +In the current versions of the code-generated .NET client, supporting commonly used endpoints is critical. Some specific queries and aggregations need further work to generate code correctly, hence they are not included yet. Ensure that the features you are using are currently supported before migrating. + +An up to date list of all supported and unsupported endpoints can be found on [GitHub](https://github.com/elastic/elasticsearch-net/issues/7890). + + +## Workarounds for missing features [_workarounds_for_missing_features] + +If you encounter a missing feature with the v8 client, there are several ways to temporarily work around this issue until we officially reintroduce the feature. + +`NEST` 7.17.x can continue to be used in [compatibility mode](https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html) with {{es}} 8.x servers until the v8 Elasticsearch .NET Client features align with application requirements. + +As a last resort, the low-level client `Elastic.Transport` can be used to create any desired request by hand: + +```csharp +public class MyRequestParameters : RequestParameters +{ + public bool Pretty + { + get => Q("pretty"); + init => Q("pretty", value); + } +} + +// ... + +var body = """ + { + "name": "my-api-key", + "expiration": "1d", + "...": "..." + } + """; + +MyRequestParameters requestParameters = new() +{ + Pretty = true +}; + +var pathAndQuery = requestParameters.CreatePathWithQueryStrings("/_security/api_key", + client.ElasticsearchClientSettings); +var endpointPath = new EndpointPath(Elastic.Transport.HttpMethod.POST, pathAndQuery); + +// Or, if the path does not contain query parameters: +// new EndpointPath(Elastic.Transport.HttpMethod.POST, "my_path") + +var response = await client.Transport + .RequestAsync( + endpointPath, + PostData.String(body), + null, + null, + cancellationToken: default) + .ConfigureAwait(false); +``` diff --git a/docs/usage/query.asciidoc b/docs/reference/query.md similarity index 64% rename from docs/usage/query.asciidoc rename to docs/reference/query.md index b365825cdbd..11f0e1d1452 100644 --- a/docs/usage/query.asciidoc +++ b/docs/reference/query.md @@ -1,13 +1,16 @@ -[[query]] -== Query examples +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/query.html +--- + +# Query examples [query] This page demonstrates how to perform a search request. -[discrete] -=== Fluent API -[source,csharp] ----- +## Fluent API [_fluent_api_3] + +```csharp var response = await client .SearchAsync(search => search .Index("persons") @@ -19,13 +22,12 @@ var response = await client ) .Size(10) ); ----- +``` -[discrete] -=== Object initializer API -[source,csharp] ----- +## Object initializer API [_object_initializer_api_3] + +```csharp var response = await client .SearchAsync(new SearchRequest("persons") { @@ -35,16 +37,15 @@ var response = await client }), Size = 10 }); ----- +``` -[discrete] -=== Consume the response +## Consume the response [_consume_the_response_3] -[source,csharp] ----- +```csharp foreach (var person in response.Documents) { Console.WriteLine(person.FirstName); } ----- +``` + diff --git a/docs/reference/recommendations.md b/docs/reference/recommendations.md new file mode 100644 index 00000000000..91a26544621 --- /dev/null +++ b/docs/reference/recommendations.md @@ -0,0 +1,25 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/recommendations.html +--- + +# Usage recommendations [recommendations] + +To achieve the most efficient use of the Elasticsearch .NET Client, we recommend following the guidance defined in this article. + + +## Reuse the same client instance [_reuse_the_same_client_instance] + +When working with the Elasticsearch .NET Client we recommend that consumers reuse a single instance of `ElasticsearchClient` for the entire lifetime of the application. When reusing the same instance: + +* initialization overhead is limited to the first usage. +* resources such as TCP connections can be pooled and reused to improve efficiency. +* serialization overhead is reduced, improving performance. + +The `ElasticsearchClient` type is thread-safe and can be shared and reused across multiple threads in consuming applications. Client reuse can be achieved by creating a singleton static instance or by registering the type with a singleton lifetime when using dependency injection containers. + + +## Prefer asynchronous methods [_prefer_asynchronous_methods] + +The Elasticsearch .NET Client exposes synchronous and asynchronous methods on the `ElasticsearchClient`. We recommend always preferring the asynchronous methods, which have the `Async` suffix. Using the Elasticsearch .NET Client requires sending HTTP requests to {{es}} servers. Access to {{es}} is sometimes slow or delayed, and some complex queries may take several seconds to return. If such operations are blocked by calling the synchronous methods, the thread must wait until the HTTP request is complete. In high-load scenarios, this can cause significant thread usage, potentially affecting the throughput and performance of consuming applications. By preferring the asynchronous methods, application threads can continue with other work that doesn’t depend on the web resource until the potentially blocking task completes. + diff --git a/docs/reference/serialization.md b/docs/reference/serialization.md new file mode 100644 index 00000000000..70b9bef8866 --- /dev/null +++ b/docs/reference/serialization.md @@ -0,0 +1,12 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/serialization.html +--- + +# Serialization [serialization] + +By default, the .NET client for {{es}} uses the Microsoft System.Text.Json library for serialization. The client understands how to serialize and deserialize the request and response types correctly. It also handles (de)serialization of user POCO types representing documents read or written to {{es}}. + +The client has two distinct serialization responsibilities - serialization of the types owned by the `Elastic.Clients.Elasticsearch` library and serialization of source documents, modeled in application code. The first responsibility is entirely internal; the second is configurable. + + diff --git a/docs/reference/source-serialization.md b/docs/reference/source-serialization.md new file mode 100644 index 00000000000..e6b110b039f --- /dev/null +++ b/docs/reference/source-serialization.md @@ -0,0 +1,524 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/source-serialization.html +--- + +# Source serialization [source-serialization] + +Source serialization refers to the process of (de)serializing POCO types in consumer applications as source documents indexed and retrieved from {{es}}. A source serializer implementation handles serialization, with the default implementation using the `System.Text.Json` library. As a result, you may use `System.Text.Json` attributes and converters to control the serialization behavior. + +* [Modelling documents with types](#modeling-documents-with-types) +* [Customizing source serialization](#customizing-source-serialization) + +## Modeling documents with types [modeling-documents-with-types] + +{{es}} provides search and aggregation capabilities on the documents that it is sent and indexes. These documents are sent as JSON objects within the request body of a HTTP request. It is natural to model documents within the {{es}} .NET client using [POCOs (*Plain Old CLR Objects*)](https://en.wikipedia.org/wiki/Plain_Old_CLR_Object). + +This section provides an overview of how types and type hierarchies can be used to model documents. + +### Default behaviour [default-behaviour] + +The default behaviour is to serialize type property names as camelcase JSON object members. + +We can model documents using a regular class (POCO). + +```csharp +public class MyDocument +{ + public string StringProperty { get; set; } +} +``` + +We can then index the an instance of the document into {{es}}. + +```csharp +using System.Threading.Tasks; +using Elastic.Clients.Elasticsearch; + +var document = new MyDocument +{ + StringProperty = "value" +}; + +var indexResponse = await Client + .IndexAsync(document, "my-index-name"); +``` + +The index request is serialized, with the source serializer handling the `MyDocument` type, serializing the POCO property named `StringProperty` to the JSON object member named `stringProperty`. + +```javascript +{ + "stringProperty": "value" +} +``` + + + +## Customizing source serialization [customizing-source-serialization] + +The built-in source serializer handles most POCO document models correctly. Sometimes, you may need further control over how your types are serialized. + +::::{note} +The built-in source serializer uses the [Microsoft `System.Text.Json` library](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/overview) internally. You can apply `System.Text.Json` attributes and converters to control the serialization of your document types. +:::: + + + +#### Using `System.Text.Json` attributes [system-text-json-attributes] + +`System.Text.Json` includes attributes that can be applied to types and properties to control their serialization. These can be applied to your POCO document types to perform actions such as controlling the name of a property or ignoring a property entirely. Visit the [Microsoft documentation for further examples](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/overview). + +We can model a document to represent data about a person using a regular class (POCO), applying `System.Text.Json` attributes as necessary. + +```csharp +using System.Text.Json.Serialization; + +public class Person +{ + [JsonPropertyName("forename")] <1> + public string FirstName { get; set; } + + [JsonIgnore] <2> + public int Age { get; set; } +} +``` + +1. The `JsonPropertyName` attribute ensures the `FirstName` property uses the JSON name `forename` when serialized. +2. The `JsonIgnore` attribute prevents the `Age` property from appearing in the serialized JSON. + + +We can then index an instance of the document into {{es}}. + +```csharp +using System; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading.Tasks; +using Elastic.Transport; +using Elastic.Clients.Elasticsearch; +using Elastic.Clients.Elasticsearch.Serialization; + +var person = new Person { FirstName = "Steve", Age = 35 }; +var indexResponse = await Client.IndexAsync(person, "my-index-name"); +``` + +The index request is serialized, with the source serializer handling the `Person` type, serializing the POCO property named `FirstName` to the JSON object member named `forename`. The `Age` property is ignored and does not appear in the JSON. + +```javascript +{ + "forename": "Steve" +} +``` + + +#### Configuring custom `JsonSerializerOptions` [configuring-custom-jsonserializeroptions] + +The default source serializer applies a set of standard `JsonSerializerOptions` when serializing source document types. In some circumstances, you may need to override some of our defaults. This is achievable by creating an instance of `DefaultSourceSerializer` and passing an `Action`, which is applied after our defaults have been set. This mechanism allows you to apply additional settings or change the value of our defaults. + +The `DefaultSourceSerializer` includes a constructor that accepts the current `IElasticsearchClientSettings` and a `configureOptions` `Action`. + +```csharp +public DefaultSourceSerializer(IElasticsearchClientSettings settings, Action configureOptions); +``` + +Our application defines the following `Person` class, which models a document we will index to {{es}}. + +```csharp +public class Person +{ + public string FirstName { get; set; } +} +``` + +We want to serialize our source document using Pascal Casing for the JSON properties. Since the options applied in the `DefaultSouceSerializer` set the `PropertyNamingPolicy` to `JsonNamingPolicy.CamelCase`, we must override this setting. After configuring the `ElasticsearchClientSettings`, we index our document to {{es}}. + +```csharp +using System; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading.Tasks; +using Elastic.Transport; +using Elastic.Clients.Elasticsearch; +using Elastic.Clients.Elasticsearch.Serialization; + +static void ConfigureOptions(JsonSerializerOptions o) => <1> + o.PropertyNamingPolicy = null; + +var nodePool = new SingleNodePool(new Uri("http://localhost:9200")); +var settings = new ElasticsearchClientSettings( + nodePool, + sourceSerializer: (defaultSerializer, settings) => + new DefaultSourceSerializer(settings, ConfigureOptions)); <2> +var client = new ElasticsearchClient(settings); + +var person = new Person { FirstName = "Steve" }; +var indexResponse = await client.IndexAsync(person, "my-index-name"); +``` + +1. A local function can be defined, accepting a `JsonSerializerOptions` parameter. Here, we set `PropertyNamingPolicy` to `null`. This returns to the default behavior for `System.Text.Json`, which uses Pascal Case. +2. When creating the `ElasticsearchClientSettings`, we supply a `SourceSerializerFactory` using a lambda. The factory function creates a new instance of `DefaultSourceSerializer`, passing in the `settings` and our `ConfigureOptions` local function. We have now configured the settings with a custom instance of the source serializer. + + +The `Person` instance is serialized, with the source serializer serializing the POCO property named `FirstName` using Pascal Case. + +```javascript +{ + "FirstName": "Steve" +} +``` + +As an alternative to using a local function, we could store an `Action` into a variable instead, which can be passed to the `DefaultSouceSerializer` constructor. + +```csharp +Action configureOptions = o => o.PropertyNamingPolicy = null; +``` + + +#### Registering custom `System.Text.Json` converters [registering-custom-converters] + +In certain more advanced situations, you may have types which require further customization during serialization than is possible using `System.Text.Json` property attributes. In these cases, the recommendation from Microsoft is to leverage a custom `JsonConverter`. Source document types serialized using the `DefaultSourceSerializer` can leverage the power of custom converters. + +For this example, our application has a document class that should use a legacy JSON structure to continue operating with existing indexed documents. Several options are available, but we’ll apply a custom converter in this case. + +Our class is defined, and the `JsonConverter` attribute is applied to the class type, specifying the type of a custom converter. + +```csharp +using System.Text.Json.Serialization; + +[JsonConverter(typeof(CustomerConverter))] <1> +public class Customer +{ + public string CustomerName { get; set; } + public CustomerType CustomerType { get; set; } +} + +public enum CustomerType +{ + Standard, + Enhanced +} +``` + +1. The `JsonConverter` attribute signals to `System.Text.Json` that it should use a converter of type `CustomerConverter` when serializing instances of this class. + + +When serializing this class, rather than include a string value representing the value of the `CustomerType` property, we must send a boolean property named `isStandard`. This requirement can be achieved with a custom JsonConverter implementation. + +```csharp +using System; +using System.Text.Json; +using System.Text.Json.Serialization; + +public class CustomerConverter : JsonConverter +{ + public override Customer Read(ref Utf8JsonReader reader, + Type typeToConvert, JsonSerializerOptions options) + { + var customer = new Customer(); + + while (reader.Read() && reader.TokenType != JsonTokenType.EndObject) + { + if (reader.TokenType == JsonTokenType.PropertyName) + { + if (reader.ValueTextEquals("customerName")) + { + reader.Read(); + customer.CustomerName = reader.GetString(); + continue; + } + + if (reader.ValueTextEquals("isStandard")) <1> + { + reader.Read(); + var isStandard = reader.GetBoolean(); + + if (isStandard) + { + customer.CustomerType = CustomerType.Standard; + } + else + { + customer.CustomerType = CustomerType.Enhanced; + } + + continue; + } + } + } + + return customer; + } + + public override void Write(Utf8JsonWriter writer, + Customer value, JsonSerializerOptions options) + { + if (value is null) + { + writer.WriteNullValue(); + return; + } + + writer.WriteStartObject(); + + if (!string.IsNullOrEmpty(value.CustomerName)) + { + writer.WritePropertyName("customerName"); + writer.WriteStringValue(value.CustomerName); + } + + writer.WritePropertyName("isStandard"); + + if (value.CustomerType == CustomerType.Standard) <2> + { + writer.WriteBooleanValue(true); + } + else + { + writer.WriteBooleanValue(false); + } + + writer.WriteEndObject(); + } +} +``` + +1. When reading, this converter reads the `isStandard` boolean and translate this to the correct `CustomerType` enum value. +2. When writing, this converter translates the `CustomerType` enum value to an `isStandard` boolean property. + + +We can then index a customer document into {{es}}. + +```csharp +using System; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading.Tasks; +using Elastic.Transport; +using Elastic.Clients.Elasticsearch; +using Elastic.Clients.Elasticsearch.Serialization; + +var customer = new Customer +{ + CustomerName = "Customer Ltd", + CustomerType = CustomerType.Enhanced +}; +var indexResponse = await Client.IndexAsync(customer, "my-index-name"); +``` + +The `Customer` instance is serialized using the custom converter, creating the following JSON document. + +```javascript +{ + "customerName": "Customer Ltd", + "isStandard": false +} +``` + + +#### Creating a custom `SystemTextJsonSerializer` [creating-custom-system-text-json-serializer] + +The built-in `DefaultSourceSerializer` includes the registration of `JsonConverter` instances which apply during source serialization. In most cases, these provide the proper behavior for serializing source documents, including those which use `Elastic.Clients.Elasticsearch` types on their properties. + +An example of a situation where you may require more control over the converter registration order is for serializing `enum` types. The `DefaultSourceSerializer` registers the `System.Text.Json.Serialization.JsonStringEnumConverter`, so enum values are serialized using their string representation. Generally, this is the preferred option for types used to index documents to {{es}}. + +In some scenarios, you may need to control the string value sent for an enumeration value. That is not directly supported in `System.Text.Json` but can be achieved by creating a custom `JsonConverter` for the `enum` type you wish to customize. In this situation, it is not sufficient to use the `JsonConverterAttribute` on the `enum` type to register the converter. `System.Text.Json` will prefer the converters added to the `Converters` collection on the `JsonSerializerOptions` over an attribute applied to an `enum` type. It is, therefore, necessary to either remove the `JsonStringEnumConverter` from the `Converters` collection or register a specialized converter for your `enum` type before the `JsonStringEnumConverter`. + +The latter is possible via several techniques. When using the {{es}} .NET library, we can achieve this by deriving from the abstract `SystemTextJsonSerializer` class. + +Here we have a POCO which uses the `CustomerType` enum as the type for a property. + +```csharp +using System.Text.Json.Serialization; + +public class Customer +{ + public string CustomerName { get; set; } + public CustomerType CustomerType { get; set; } +} + +public enum CustomerType +{ + Standard, + Enhanced +} +``` + +To customize the strings used during serialization of the `CustomerType`, we define a custom `JsonConverter` specific to our `enum` type. + +```csharp +using System.Text.Json.Serialization; + +public class CustomerTypeConverter : JsonConverter +{ + public override CustomerType Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + { + return reader.GetString() switch <1> + { + "basic" => CustomerType.Standard, + "premium" => CustomerType.Enhanced, + _ => throw new JsonException( + $"Unknown value read when deserializing {nameof(CustomerType)}."), + }; + } + + public override void Write(Utf8JsonWriter writer, CustomerType value, JsonSerializerOptions options) + { + switch (value) <2> + { + case CustomerType.Standard: + writer.WriteStringValue("basic"); + return; + case CustomerType.Enhanced: + writer.WriteStringValue("premium"); + return; + } + + writer.WriteNullValue(); + } +} +``` + +1. When reading, this converter translates the string used in the JSON to the matching enum value. +2. When writing, this converter translates the `CustomerType` enum value to a custom string value written to the JSON. + + +We create a serializer derived from `SystemTextJsonSerializer` to give us complete control of converter registration order. + +```csharp +using System.Text.Json; +using Elastic.Clients.Elasticsearch.Serialization; + +public class MyCustomSerializer : SystemTextJsonSerializer <1> +{ + private readonly JsonSerializerOptions _options; + + public MyCustomSerializer(IElasticsearchClientSettings settings) : base(settings) + { + var options = DefaultSourceSerializer.CreateDefaultJsonSerializerOptions(false); <2> + + options.Converters.Add(new CustomerTypeConverter()); <3> + + _options = DefaultSourceSerializer.AddDefaultConverters(options); <4> + } + + protected override JsonSerializerOptions CreateJsonSerializerOptions() => _options; <5> +} +``` + +1. Inherit from `SystemTextJsonSerializer`. +2. In the constructor, use the factory method `DefaultSourceSerializer.CreateDefaultJsonSerializerOptions` to create default options for serialization. No default converters are registered at this stage because we pass `false` as an argument. +3. Register our `CustomerTypeConverter` as the first converter. +4. To apply any default converters, call the `DefaultSourceSerializer.AddDefaultConverters` helper method, passing the options to modify. +5. Implement the `CreateJsonSerializerOptions` method returning the stored `JsonSerializerOptions`. + + +Because we have registered our `CustomerTypeConverter` before the default converters (which include the `JsonStringEnumConverter`), our converter takes precedence when serializing `CustomerType` instances on source documents. + +The base `SystemTextJsonSerializer` class handles the implementation details of binding, which is required to ensure that the built-in converters can access the `IElasticsearchClientSettings` where needed. + +We can then index a customer document into {{es}}. + +```csharp +using System; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading.Tasks; +using Elastic.Transport; +using Elastic.Clients.Elasticsearch; +using Elastic.Clients.Elasticsearch.Serialization; + +var customer = new Customer +{ + CustomerName = "Customer Ltd", + CustomerType = CustomerType.Enhanced +}; + +var indexResponse = await client.IndexAsync(customer, "my-index-name"); +``` + +The `Customer` instance is serialized using the custom `enum` converter, creating the following JSON document. + +```javascript +{ + "customerName": "Customer Ltd", + "customerType": "premium" <1> +} +``` + +1. The string value applied during serialization is provided by our custom converter. + + + +#### Creating a custom `Serializer` [creating-custom-serializers] + +Suppose you prefer using an alternative JSON serialization library for your source types. In that case, you can inject an isolated serializer only to be called for the serialization of `_source`, `_fields`, or wherever a user-provided value is expected to be written and returned. + +Implementing `Elastic.Transport.Serializer` is technically enough to create a custom source serializer. + +```csharp +using System; +using System.IO; +using System.Threading; +using System.Threading.Tasks; +using Elastic.Transport; + +public class VanillaSerializer : Serializer +{ + public override object Deserialize(Type type, Stream stream) => + throw new NotImplementedException(); + + public override T Deserialize(Stream stream) => + throw new NotImplementedException(); + + public override ValueTask DeserializeAsync(Type type, Stream stream, CancellationToken cancellationToken = default) => + throw new NotImplementedException(); + + public override ValueTask DeserializeAsync(Stream stream, CancellationToken cancellationToken = default) => + throw new NotImplementedException(); + + public override void Serialize(T data, Stream stream, SerializationFormatting formatting = SerializationFormatting.None) => + throw new NotImplementedException(); + + public override Task SerializeAsync(T data, Stream stream, + SerializationFormatting formatting = SerializationFormatting.None, CancellationToken cancellationToken = default) => + throw new NotImplementedException(); +} +``` + +Registering up the serializer is performed in the `ConnectionSettings` constructor. + +```csharp +using System; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Threading.Tasks; +using Elastic.Transport; +using Elastic.Clients.Elasticsearch; +using Elastic.Clients.Elasticsearch.Serialization; + +var nodePool = new SingleNodePool(new Uri("http://localhost:9200")); +var settings = new ElasticsearchClientSettings( + nodePool, + sourceSerializer: (defaultSerializer, settings) => + new VanillaSerializer()); <1> +var client = new ElasticsearchClient(settings); +``` + +1. If implementing `Serializer` is enough, why must we provide an instance wrapped in a factory `Func`? + + +There are various cases where you might have a POCO type that contains an `Elastic.Clients.Elasticsearch` type as one of its properties. The `SourceSerializerFactory` delegate provides access to the default built-in serializer so you can access it when necessary. For example, consider if you want to use percolation; you need to store {{es}} queries as part of the `_source` of your document, which means you need to have a POCO that looks like this. + +```csharp +using Elastic.Clients.Elasticsearch.QueryDsl; + +public class MyPercolationDocument +{ + public Query Query { get; set; } + public string Category { get; set; } +} +``` + +A custom serializer would not know how to serialize `Query` or other `Elastic.Clients.Elasticsearch` types that could appear as part of the `_source` of a document. Therefore, your custom `Serializer` would need to store a reference to our built-in serializer and delegate serialization of Elastic types back to it. + + diff --git a/docs/reference/toc.yml b/docs/reference/toc.yml new file mode 100644 index 00000000000..e8e6d33012e --- /dev/null +++ b/docs/reference/toc.yml @@ -0,0 +1,23 @@ +toc: + - file: index.md + - file: getting-started.md + - file: installation.md + - file: connecting.md + - file: configuration.md + children: + - file: _options_on_elasticsearchclientsettings.md + - file: client-concepts.md + children: + - file: serialization.md + children: + - file: source-serialization.md + - file: using-net-client.md + children: + - file: aggregations.md + - file: esql.md + - file: examples.md + - file: mappings.md + - file: query.md + - file: recommendations.md + - file: transport.md + - file: migration-guide.md \ No newline at end of file diff --git a/docs/usage/transport.asciidoc b/docs/reference/transport.md similarity index 79% rename from docs/usage/transport.asciidoc rename to docs/reference/transport.md index 3e15fbd0b90..8af799958ab 100644 --- a/docs/usage/transport.asciidoc +++ b/docs/reference/transport.md @@ -1,10 +1,13 @@ -[[transport]] -== Transport example +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/transport.html +--- + +# Transport example [transport] This page demonstrates how to use the low level transport to send requests. -[source,csharp] ----- +```csharp public class MyRequestParameters : RequestParameters { public bool Pretty @@ -19,7 +22,7 @@ public class MyRequestParameters : RequestParameters var body = """ { "name": "my-api-key", - "expiration": "1d", + "expiration": "1d", "...": "..." } """; @@ -33,7 +36,7 @@ var pathAndQuery = requestParameters.CreatePathWithQueryStrings("/_security/api_ client.ElasticsearchClientSettings); var endpointPath = new EndpointPath(Elastic.Transport.HttpMethod.POST, pathAndQuery); -// Or, if the path does not contain query parameters: +// Or, if the path does not contain query parameters: // new EndpointPath(Elastic.Transport.HttpMethod.POST, "my_path") var response = await client.Transport @@ -44,4 +47,5 @@ var response = await client.Transport null, cancellationToken: default) .ConfigureAwait(false); ----- +``` + diff --git a/docs/reference/using-net-client.md b/docs/reference/using-net-client.md new file mode 100644 index 00000000000..7d68c0bccb2 --- /dev/null +++ b/docs/reference/using-net-client.md @@ -0,0 +1,24 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/usage.html +--- + +# Using the .NET Client [usage] + +The sections below provide tutorials on the most frequently used and some less obvious features of {{es}}. + +For a full reference, see the [Elasticsearch documentation](docs-content://get-started/index.md) and in particular the [REST APIs](elasticsearch://docs/reference/elasticsearch/rest-apis/index.md) section. The Elasticsearch .NET Client follows closely the JSON structures described there. + +A .NET API reference documentation for the Elasticsearch client package is available [here](https://elastic.github.io/elasticsearch-net). + +If you’re new to {{es}}, make sure also to read [Elasticsearch’s quick start](docs-content://solutions/search/get-started.md) that provides a good introduction. + +* [Usage recommendations](/reference/recommendations.md) +* [CRUD usage examples](/reference/examples.md) +* [Using ES|QL](/reference/esql.md) + +::::{note} +This is still a work in progress, more sections will be added in the near future. +:::: + + diff --git a/docs/release-notes/breaking-change-policy.asciidoc b/docs/release-notes/breaking-change-policy.asciidoc deleted file mode 100644 index 74138bc005f..00000000000 --- a/docs/release-notes/breaking-change-policy.asciidoc +++ /dev/null @@ -1,32 +0,0 @@ -[[breaking-changes-policy]] -== Breaking changes policy - -The {net-client} source code is generated from a https://github.com/elastic/elasticsearch-specification[formal specification of the Elasticsearch API]. This API specification is large, and although it is tested against hundreds of Elasticsearch test files, it may have discrepancies with the actual API that result in issues in the {net-client}. - -Fixing these discrepancies in the API specification results in code changes in the {net-client}, and some of these changes can require code updates in your applications. - -This section explains how these breaking changes are considered for inclusion in {net-client} releases. - -[discrete] -==== Breaking changes in patch releases - -Some issues in the API specification are properties that have an incorrect type, such as a `long` that should be a `string`, or a required property that is actually optional. These issues can cause the {net-client} to not work properly or even throw exceptions. - -When a specification issue is discovered and resolved, it may require code updates in applications using the {net-client}. Such breaking changes are considered acceptable, _even in patch releases_ (e.g. 8.0.0 -> 8.0.1), as they introduce stability to APIs that may otherwise be unusable. - -We may also make breaking changes in patch releases to correct design flaws and code-generation issues that we deem beneficial to resolve at the earliest oppotunity. We will detail these in the relevant release notes and limit these as the client matures. - -[discrete] -==== Breaking changes in minor releases - -Along with these bug fixes, the API specification is constantly refined, more precise type definitions are introduced to improve developer comfort and remove ambiguities. The specification of often-used APIs is fairly mature, so these changes happen generally on less often used APIs. These changes can also cause breaking changes requiring code updates which are considered _acceptable in minor releases_ (e.g. 8.0 -> 8.1). - -[discrete] -==== Breaking changes in major releases - -Major releases (e.g. 7.x -> 8.x) can include larger refactorings of the API specification and the framework underlying the {net-client}. These refactorings are considered carefully and done only when they unlock new important features or new developments. - -[discrete] -==== Elasticsearch API stability guarantees - -All Elasticsearch APIs have stability indicators, which imply potential changes. If an API is `stable` only additional non-breaking changes are added. In case of `experimental` APIs, breaking changes can be introduced any time, which means that these changes, will also be reflected in the {net-client}. \ No newline at end of file diff --git a/docs/release-notes/breaking-changes.md b/docs/release-notes/breaking-changes.md new file mode 100644 index 00000000000..625089862bd --- /dev/null +++ b/docs/release-notes/breaking-changes.md @@ -0,0 +1,28 @@ +--- +navigation_title: "Elasticsearch .NET Client" +--- + +# Elasticsearch .NET Client breaking changes [elasticsearch-net-client-breaking-changes] +Before you upgrade, carefully review the Elasticsearch .NET Client breaking changes and take the necessary steps to mitigate any issues. + +To learn how to upgrade, check out . + +% ## Next version [elasticsearch-nett-client-nextversion-breaking-changes] +% **Release date:** Month day, year + +% ::::{dropdown} Title of breaking change +% Description of the breaking change. +% For more information, check [PR #](PR link). +% **Impact**
Impact of the breaking change. +% **Action**
Steps for mitigating deprecation impact. +% :::: + +% ## 9.0.0 [elasticsearch-nett-client-900-breaking-changes] +% **Release date:** March 25, 2025 + +% ::::{dropdown} Title of breaking change +% Description of the breaking change. +% For more information, check [PR #](PR link). +% **Impact**
Impact of the breaking change. +% **Action**
Steps for mitigating deprecation impact. +% :::: \ No newline at end of file diff --git a/docs/release-notes/deprecations.md b/docs/release-notes/deprecations.md new file mode 100644 index 00000000000..aac4cf32899 --- /dev/null +++ b/docs/release-notes/deprecations.md @@ -0,0 +1,26 @@ +--- +navigation_title: "Elasticsearch .NET Client" +--- + +# Elasticsearch .NET Client deprecations [elasticsearch-net-client-deprecations] +Review the deprecated functionality for your Elasticsearch .NET Client version. While deprecations have no immediate impact, we strongly encourage you update your implementation after you upgrade. + +To learn how to upgrade, check out . + +% ## Next version [elasticsearch-net-client-versionnext-deprecations] +% **Release date:** Month day, year + +% ::::{dropdown} Deprecation title +% Description of the deprecation. +% For more information, check [PR #](PR link). +% **Impact**
Impact of deprecation. +% **Action**
Steps for mitigating deprecation impact. +% :::: + +% ## 9.0.0 [elasticsearch-net-client-900-deprecations] +% **Release date:** March 25, 2025 + +% Description of the deprecation and steps to update implementation. +% For more information, check [PR #](PR link). +% **Impact**
+% **Action**
\ No newline at end of file diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md new file mode 100644 index 00000000000..eeba1cc21eb --- /dev/null +++ b/docs/release-notes/index.md @@ -0,0 +1,29 @@ +--- +navigation_title: "Elasticsearch .NET Client" +mapped_pages: + - https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/release-notes.html +--- + +# Elasticsearch .NET Client release notes [elasticsearch-net-client-release-notes] + +Review the changes, fixes, and more in each version of Elasticsearch .NET Client. + +To check for security updates, go to [Security announcements for the Elastic stack](https://discuss.elastic.co/c/announcements/security-announcements/31). + +% Release notes include only features, enhancements, and fixes. Add breaking changes, deprecations, and known issues to the applicable release notes sections. + +% ## version.next [felasticsearch-net-client-next-release-notes] +% **Release date:** Month day, year + +% ### Features and enhancements [elasticsearch-net-client-next-features-enhancements] +% * + +% ### Fixes [elasticsearch-net-client-next-fixes] +% * + +## 9.0.0 [elasticsearch-net-client-900-release-notes] +**Release date:** March 25, 2025 + +### Features and enhancements [elasticsearch-net-client-900-features-enhancements] + +### Fixes [elasticsearch-net-client-900-fixes] \ No newline at end of file diff --git a/docs/release-notes/known-issues.md b/docs/release-notes/known-issues.md new file mode 100644 index 00000000000..3c227ec96cd --- /dev/null +++ b/docs/release-notes/known-issues.md @@ -0,0 +1,20 @@ +--- +navigation_title: "Elasticsearch .NET Client" + +--- + +# Elasticsearch .NET Client known issues [elasticsearch-net-client-known-issues] + +% Use the following template to add entries to this page. + +% :::{dropdown} Title of known issue +% **Details** +% On [Month/Day/Year], a known issue was discovered that [description of known issue]. + +% **Workaround** +% Workaround description. + +% **Resolved** +% On [Month/Day/Year], this issue was resolved. + +::: \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.0.0.asciidoc b/docs/release-notes/release-notes-8.0.0.asciidoc deleted file mode 100644 index 5ab8ff266fc..00000000000 --- a/docs/release-notes/release-notes-8.0.0.asciidoc +++ /dev/null @@ -1,511 +0,0 @@ -[[release-notes-8.0.0]] -== Release notes v8.0.0 - -[TIP] --- -Due to the extensive changes in the new {net-client}, we highly recommend -reviewing this documentation in full, before proceeding with migration to v8. --- - -After many months of work, eleven alphas, six betas and two release candidates, -we are pleased to announce the GA release of the {net-client} v8.0.0. - -The overall themes of this release have been based around redesigning the client -for the future, standardizing serialization, performance improvements, codebase -simplification, and code-generation. - -The following release notes explain the current state of the client, missing -features, breaking changes and our rationale for some of the design choices we have introduced. - -[discrete] -=== Version 8 is a refresh - -[IMPORTANT] --- -It is important to highlight that v8 of the {net-client} represents -a new start for the client design. It is important to review how this may affect -your code and usage. --- - -Mature code becomes increasingly hard to maintain over time, and -our ability to make timely releases has diminished as code complexity has increased. -Major releases allow us to simplify and better align our language clients with -each other in terms of design. Here, it is crucial to find the right balance -between uniformity across programming languages and the idiomatic concerns of -each language. For .NET, we will typically compare and contrast with https://github.com/elastic/elasticsearch-java[Java] and https://github.com/elastic/go-elasticsearch[Go] -to make sure that our approach is equivalent for each of these. We also take -heavy inspiration from Microsoft framework design guidelines and the conventions -of the wider .NET community. - -[discrete] -==== New Elastic.Clients.Elasticsearch NuGet package - -We have shipped the new code-generated client as a -https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/[new NuGet package] -with a new root namespace, `Elastic.Clients.Elasticsearch`. -The new v8 client is built upon the foundations of the v7 `NEST` client, but there -are changes. By shipping as a new package, the expectation is that migration can -be managed with a phased approach. - -While this is a new package, we have aligned the major version (v8.x.x) with the -supported {es} server version to clearly indicate the client/server compatibility. -The v8 client is designed to work with version 8 of {es}. - -The v7 `NEST` client continues to be supported but will not gain new features or -support for new {es} endpoints. It should be considered deprecated in favour of -the new client. - -[discrete] -==== Limited feature set - -[CAUTION] --- -The 8.0.0 {net-client} does not have feature parity with the previous v7 `NEST` -high-level client. --- - -For the initial 8.0.0 release we have limited the features we are shipping. -Over the course of the 8.x releases, we will reintroduce features. Therefore, -if something is missing, it may not be permanently omitted. We will endeavour to communicate our plans as and when they become available. - -If a feature you depend on is missing (and not explicitly documented below as a -feature that we do not plan to reintroduce), please open https://github.com/elastic/elasticsearch-net/issues/new/choose[an issue] -or comment on a relevant existing issue to highlight your need to us. This will -help us prioritise our roadmap. - -[discrete] -=== Code generation - -Given the size of the Elasticsearch API surface today, it is no longer practical -to maintain thousands of types (requests, responses, queries, aggregations, etc.) -by hand. To ensure consistent, accurate and timely alignment between language -clients and {es}, the 8.x clients, and many of the associated types are now -automatically code-generated from a https://github.com/elastic/elasticsearch-specification[shared specification]. This is a common solution to maintaining alignment between -client and server among SDKs and libraries, such as those for Azure, AWS and the -Google Cloud Platform. - -Code-generation from a specification has inevitably led to some differences -between the existing v7 `NEST` types and those available in the new v7 {net-client}. -For the 8.0.0 release, we generate strictly from the specification, special -casing a few areas to improve usability or to align with language idioms. - -The base type hierarchy for concepts such as `Properties`, `Aggregations` and -`Queries` is no longer present in generated code, as these arbitrary groupings do -not align with concrete concepts of the public server API. These considerations -do not preclude adding syntactic sugar and usability enhancements to types in future -releases on a case-by-case basis. - -[discrete] -=== Elastic.Transport - -The .NET client includes a transport layer responsible for abstracting HTTP -concepts and to provide functionality such as our request pipeline. This -supports round-robin load-balancing of requests to nodes, pinging failed -nodes and sniffing the cluster for node roles. - -In v7, this layer shipped as `Elasticsearch.Net` and was considered our low-level -client which could be used to send and receive raw JSON bytes between the client -and server. - -As part of the work for 8.0.0, we have moved the transport layer out into -a https://www.nuget.org/packages/Elastic.Transport[new dedicated package] and -https://github.com/elastic/elastic-transport-net[repository], named -`Elastic.Transport`. This supports reuse across future clients and allows -consumers with extremely high-performance requirements to build upon this foundation. - -[discrete] -=== System.Text.Json for serialization - -The v7 `NEST` high-level client used an internalized and modified version of -https://github.com/neuecc/Utf8Json[Utf8Json] for request and response -serialization. This was introduced for its performance improvements -over https://www.newtonsoft.com/json[Json.NET], the more common JSON framework at -the time. - -While Utf8Json provides good value, we have identified minor bugs and -performance issues that have required maintenance over time. Some of these -are hard to change without more significant effort. This library is no longer -maintained, and any such changes cannot easily be contributed back to the -original project. - -With .NET Core 3.0, Microsoft shipped new https://devblogs.microsoft.com/dotnet/try-the-new-system-text-json-apis[System.Text.Json APIs] -that are included in-the-box with current versions of .NET. We have adopted -`System.Text.Json` for all serialization. Consumers can still define and register -their own `Serializer` implementation for their document types should they prefer -to use a different serialization library. - -By adopting `System.Text.Json`, we now depend on a well-maintained and supported -library from Microsoft. `System.Text.Json` is designed from the ground up to support -the latest performance optimizations in .NET and, as a result, provides both fast and low-allocation serialization. - -[discrete] -=== Mockability of ElasticsearchClient - -Testing code is an important part of software development. We recommend -that consumers prefer introducing an abstraction for their use of the {net-client} -as the prefered way to decouple consuming code from client types and support unit -testing. - -In order to support user testing scenarios, we have unsealed the `ElasticsearchClient` -type and made its methods virtual. This supports mocking the type directly for unit -testing. This is an improvement over the original `IElasticClient` interface from -`NEST` (v7) which only supported mocking of top-level client methods. - -We have also introduced a `TestableResponseFactory` in `Elastic.Transport` to -make it easier to create response instances with specific status codes and validity -that can be used during unit testing. - -These changes are in addition to our existing support for testing with an -`InMemoryConnection`, virtualized clusters and with our -https://github.com/elastic/elasticsearch-net-abstractions/blob/master/src/Elastic.Elasticsearch.Managed[`Elastic.Elasticsearch.Managed`] library for integration -testing against real {es} instances. We will introduce more documentation on testing methodologies in the near future. - -[discrete] -=== Migrating to Elastic.Clients.Elasticsearch - -[WARNING] --- -The 8.0.0 release does not currently have full-feature parity with `NEST`. The -client primary use case is for application developers communitating with {es}. --- - -The 8.0.0 release focuses on core endpoints, more specifically for common CRUD -scenarios. We intend to reduce the feature gap in subsequent versions. We anticipate -that this initial release will best suit new applications and may not yet be migration-ready for all existing consumers. We recommend reviewing this documentation carefully to learn about the missing features and reduced API surface details before migrating from the v7 `NEST` client. - -The choice to code-generate a new evolution of the {net-client} introduces some -significant breaking changes. We consciously took the opportunity to refactor -and reconsider historic design choices as part of this major release, intending -to limit future breaking changes. - -The v8 client is shipped as a new https://www.nuget.org/packages/Elastic.Clients.Elasticsearch/[NuGet package] -which can be installed alongside v7 `NEST`. We -anticipate that some consumers may prefer a phased migration with both -packages side-by-side for a short period of time to manage complex migrations. In addition, `NEST` 7.17.x can continue to be used in -https://www.elastic.co/guide/en/elasticsearch/client/net-api/7.17/connecting-to-elasticsearch-v8.html[compatibility mode] -with {es} 8.x servers until the v8 {net-client} features -align with application requirements. - -We will continue to prioritize the feature roadmap and code-generation work -based on https://github.com/elastic/elasticsearch-net/issues[feedback] from consumers who may rely on features that are initially unavailable. - -[discrete] -=== Breaking Changes - -[WARNING] --- -As a result of code-generating a majority of the client types, this version of -the client includes multiple breaking changes. --- - -We have strived to keep the core foundation reasonably similar, but types emitted -through code-generation are subject to change between `NEST` (v7) and the new -`Elastic.Clients.Elasticsearch` (v8) package. - -[discrete] -==== Namespaces - -We have renamed the package and top-level namespace for the v8 client to -`Elastic.Clients.Elasticsearch`. All types belong to this namespace. When -necessary, to avoid potential conflicts, types are generated into suitable -sub-namespaces based on the https://github.com/elastic/elasticsearch-specification[{es} specification]. Additional `using` directives may be required to access such types -when using the {net-client}. - -Transport layer concepts have moved to the new `Elastic.Transport` NuGet package -and related types are defined under its namespace. Some configuration and low-level transport functionality may require a `using` directive for the `Elastic.Transport` -namespace. - -[discrete] -==== Type names - -Type names may have changed from previous versions. We are not listing these -explicitly due to the potentially vast number of subtle differences. -Type names will now more closely align to those used in the JSON and as documented -in the {es} documentation. - -[discrete] -==== Class members - -Types may include renamed properties based on the {es} specification, -which differ from the original `NEST` property names. The types used for properties -may also have changed due to code-generation. If you identify missing or -incorrectly-typed properties, please open https://github.com/elastic/elasticsearch-net/issues/new/choose[an issue] to alert us. - -[discrete] -==== Sealing classes - -Opinions on "sealing by default" within the .NET ecosystem tend to be quite -polarized. Microsoft seal all internal types for potential performance gains -and we see a benefit in starting with that approach for the {net-client}, -even for our public API surface. - -While it prevents inheritance and, therefore, may inhibit a few consumer scenarios, -sealing by default is intended to avoid the unexpected or invalid -extension of types that could inadvertently be broken in the future. That said, -sealing is not necessarily a final choice for all types; but it is clearly -easier for a future release to unseal a previously-sealed class than -vice versa. We can therefore choose to unseal as valid scenarios arise, -should we determine that doing so is the best solution for those scenarios, such -as with mockability of the `ElasticsearchClient`. This goes back to our clean-slate concept for this new client. - -[discrete] -==== Removed features - -As part of the clean-slate redesign of the new client, we have opted to remove -certain features from the v8.0 client. These are listed below: - -[discrete] -===== Attribute mappings - -In previous versions of the `NEST` client, attributes could be used to configure -the mapping behaviour and inference for user types. We have removed support for -these attributes and recommend that mapping be completed via the fluent API when -configuring client instances. `System.Text.Json` attributes may be used to rename -and ignore properties during source serialization. - -[discrete] -===== CAT APIs - -The https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html[CAT APIs] -of {es} are intended for human-readable usage and will no longer be supported -via the v8 {net-client}. - -[discrete] -===== Interface removal - -We have removed several interfaces that previously shipped as part of `NEST`. This -is a design decision to simplify the library and avoid interfaces where only a -single implementation of that interface is expected to exist, such as -`IElasticClient` in `NEST`. We have also switched to prefer abstract base classes -over interfaces across the library, as this allows us to add enhancements more -easily without introducing breaking changes for derived types. - -[discrete] -==== Missing features - -While not an exhaustive list, the following are some of the main features which -have not been re-implemented for this initial 8.0.0 GA release. -These remain on our roadmap and will be reviewed and prioritized for inclusion in -future releases. - -* Query DSL operators for combining queries. -* Scroll Helper. -* Fluent API for union types. -* `AutoMap` for field datatype inference. -* Visitor pattern support for types such as `Properties`. -* Support for `JoinField` which affects `ChildrenAggregation`. -* Conditionless queries. -* DiagnosticSources have been removed in `Elastic.Transport` to provide a clean-slate -for an improved diagnostics story. The {net-client} emits https://opentelemetry.io/[OpenTelemetry] compatible `Activity` spans which can be consumed by APM agents such as the https://www.elastic.co/guide/en/apm/agent/dotnet/current/index.html[Elastic APM Agent for .NET]. -* Documentation is a work in progress, and we will expand on the documented scenarios -in future releases. - -[discrete] -=== Reduced API surface - -In this first release of the code-generated .NET client, we have specifically -focused on supporting commonly used endpoints. We have also skipped specific -queries and aggregations which need further work to generate code correctly. -Before migrating, please refer to the lists below for the endpoints, -queries and aggregations currently generated and available -in the 8.0.0 GA release to ensure that the features you are using are currently -supported. - -[discrete] -==== Supported {es} endpoints - -The following are {es} endpoints currently generated and available -in the 8.0.0 {net-client}. - -* AsyncSearch.Delete -* AsyncSearch.Get -* AsyncSearch.Status -* AsyncSearch.Submit -* Bulk -* ClearScroll -* ClosePointInTime -* Cluster.Health -* Count -* Create -* Delete -* DeleteByQuery -* DeleteByQueryRethrottle -* DeleteScript -* EQL.Delete -* EQL.Get -* EQL.Search -* EQL.Status -* Exists -* ExistsSource -* Explain -* FieldCaps -* Get -* GetScript -* GetScriptContext -* GetScriptLanguages -* GetSource -* Index -* Indices.Clone -* Indices.Close -* Indices.Create -* Indices.CreateDataStream -* Indices.Delete -* Indices.DeleteAlias -* Indices.DeleteDataStream -* Indices.DeleteIndexTemplate -* Indices.DeleteTemplate -* Indices.Exists -* Indices.ExistsIndexTemplate -* Indices.ExistsTemplate -* Indices.Flush -* Indices.ForceMerge -* Indices.Get -* Indices.GetAlias -* Indices.GetDataStream -* Indices.GetFieldMapping -* Indices.GetIndexTemplate -* Indices.GetMapping -* Indices.GetTemplate -* Indices.Indices.SimulateTemplate -* Indices.MigrateToDataStream -* Indices.Open -* Indices.PutAlias -* Indices.PutIndexTemplate -* Indices.PutMapping -* Indices.PutTemplate -* Indices.Refresh -* Indices.Rollover -* Indices.Shrink -* Indices.SimulateIndexTemplate -* Indices.Split -* Indices.Unfreeze -* Info -* MGet -* MSearch -* MSearchTemplate -* OpenPointInTime -* Ping -* PutScript -* RankEval -* Reindex -* ReindexRethrottle -* Scroll -* Search -* SearchShards -* SQL.ClearCursor -* SQL.DeleteAsync -* SQL.GetAsync -* SQL.GetAsyncStatus -* SQL.Query -* TermsEnum -* Update -* UpdateByQuery -* UpdateByQueryRethrottle - -[discrete] -==== Supported queries - -The following are query types currently generated and available -in the 8.0.0 {net-client}. - -* Bool -* Boosting -* CombinedFields -* ConstantScore -* DisMax -* Exists -* FunctionScore -* Fuzzy -* HasChild -* HasParent -* Ids -* Intervals -* Match -* MatchAll -* MatchBool -* MatchNone -* MatchPhrase -* MatchPhrasePrefix -* MoreLikeThis -* MultiMatch -* Nested -* ParentId -* Percolate -* Prefix -* QueryString -* RankFeature -* Regexp -* Script -* ScriptScore -* Shape -* SimpleQueryString -* SpanContaining -* SpanFirst -* SpanMultiTerm -* SpanNear -* SpanNot -* SpanOr -* SpanTerm -* SpanWithin -* Term -* Terms -* TermsSet -* Wildcard -* Wrapper - -[discrete] -==== Supported aggregations - -The following are aggregation types currently generated and available -in the 8.0.0 {net-client}. - -* AdjacencyMatrix -* AutoDateHistogram -* Avg -* Boxplot -* Cardinality -* Children -* Composite -* CumulativeCardinality -* DateHistogram -* DateRange -* Derivative -* ExtendedStats -* Filters -* Global -* Histogram -* Inference -* IpRange -* MatrixStats -* Max -* MedianAbsoluteDeviation -* Min -* Missing -* MultiTerms -* Nested -* Parent -* PercentilesBucket -* Range -* Rate -* ReverseNested -* Sampler -* ScriptedMetric -* Stats -* StatsBucket -* StringStats -* Sum -* Terms -* TopHits -* TopMetrics -* TTest -* ValueCount -* VariableWidthHistogram -* WeightedAvg - -[discrete] -=== In closing - -Please give the new `Elastic.Clients.Elasticsearch` client a try in your .NET -applications. If you run into any problems, please open https://github.com/elastic/elasticsearch-net/issues/new/choose[an issue] to raise those -with us. Please let us know how you get on and if you have any questions, -reach out on the https://discuss.elastic.co[Discuss forums]. \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.0.1.asciidoc b/docs/release-notes/release-notes-8.0.1.asciidoc deleted file mode 100644 index 93d2dd8d376..00000000000 --- a/docs/release-notes/release-notes-8.0.1.asciidoc +++ /dev/null @@ -1,71 +0,0 @@ -[[release-notes-8.0.1]] -== Release notes v8.0.1 - -[discrete] -=== Bug fixes - -- Fix MultiSearchTemplateRequest body serialization (issue: -https://github.com/elastic/elasticsearch-net/issues/7006[#7006]) - -[discrete] -=== Enhancements - -- Seal union types for consistency - -[discrete] -=== Breaking changes - -This release includes the following breaking changes: - -[discrete] -==== MultiSearchTemplate type changes - -The `Core.MSearchTemplate.RequestItem` type has been renamed to -`Core.MSearchTemplate.SearchTemplateRequestItem`. It no longer derives from the -`Union` type. It has been manually designed to support serialization to -NDJSON, as required by the MSearchTemplate endpoint. - -The `MultiSearchTemplateRequest.SearchTemplates` property has been updated to -use this newly defined type. - -This breaking change has been included in this patch release due to the -original code-generated type functioning incorrectly, and therefore, we have -determined that this should ship ASAP. - -[discrete] -==== MultiSearch type changes - -The `Core.MSearch.SearchRequestItem` type has been sealed for consistency with -the design choices of the rest of the client. While technically breaking, we -have decided that this should be included in this release before any potentially -derived types may exist in consuming applications. - -[discrete] -==== Sealing union types - -Code-generated types derived from `Union` were incorrectly unsealed. -While technically breaking, we have decided that these should be sealed in this -patch release before any potential derived types may exist in consuming -applications. Sealing types by default aligns with our broader design choices -and this decision is described in the <>. - -Affected types: -- `Aggregations.Buckets` -- `Aggregations.FieldDateMatch` -- `Aggregations.Percentiles` -- `Analysis.CharFilter` -- `Analysis.TokenFilter` -- `Analysis.Tokenizer` -- `ByteSize` -- `Fuzziness` -- `GeoHashPrecision` -- `MultiGetResponseItem` -- `MultiSearchResponseItem` -- `QueryDsl.Like` -- `QueryDsl.TermsQueryField` -- `Script` -- `Slices` -- `SourceConfig` -- `SourceConfigParam` -- `Tasks.TaskInfos` -- `TrackHits` \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.0.10.asciidoc b/docs/release-notes/release-notes-8.0.10.asciidoc deleted file mode 100644 index 8bd7bfd896e..00000000000 --- a/docs/release-notes/release-notes-8.0.10.asciidoc +++ /dev/null @@ -1,11 +0,0 @@ -[[release-notes-8.0.10]] -== Release notes v8.0.10 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7549[#7549] Update to latest -transport to ensure ActivitySource is static. (issue: https://github.com/elastic/elasticsearch-net/issues/7540[#7540]) - -This avoids undue and potentially high volume allocations of `ActivitySource` across -consuming applications and is therefore a recommended upgrade. \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.0.2.asciidoc b/docs/release-notes/release-notes-8.0.2.asciidoc deleted file mode 100644 index 5b53dcf4773..00000000000 --- a/docs/release-notes/release-notes-8.0.2.asciidoc +++ /dev/null @@ -1,82 +0,0 @@ -[[release-notes-8.0.2]] -== Release notes v8.0.2 - -[discrete] -=== Bug fixes - -- Add missing accessor properties for dictionary responses (issue: -https://github.com/elastic/elasticsearch-net/issues/7048[#7048]) -- Fix to ensure dynamic HTTP methods are used when available (issue: -https://github.com/elastic/elasticsearch-net/issues/7057[#7057]) -- Fix resolvable dictionary properties (issue: -https://github.com/elastic/elasticsearch-net/issues/7075[#7075]) - -[discrete] -=== Breaking changes - -Some low-impact changes were made to existing types to fix the resolvable -dictionary properties. We determined it worthwhile to retype the properties to -prefer the interfaces over concrete types. - -[discrete] -==== Changes to dictionary properties on generated types - -As part of fixing the resolvable dictionary properties some low-impact changes -were made to the generated types. We determined it worthwhile to retype the -properties to prefer the interfaces over concrete types. - -Types that are immutable and only apply to server responses now use -`IReadOnlyDictionary` for relevant properties. For mutable types, we prefer -`IDictionary`. - -`HealthResponse.Indices` has changed from a bespoke `ReadOnlyIndexNameDictionary` -property to prefer `IReadOnlyDictionary` to improve ease of use and familiarity. - -[discrete] -==== Internalise ReadOnlyIndexNameDictionary - -After changes for resolvable dictionaries, the `ReadOnlyIndexNameDictionary` type -was made internal and is no longer part of the public API. Properties that -previously used this type are now typed as `IReadOnlyDictionary`. This brings -advantages in being more familiar for developers. - -[discrete] -==== Remove IndexName.GetString(ITransportConfiguration settings) method - -This method is used internally by the client and should not be exposed to -consuming applications. Instead, we prefer explicit interface implementation for -`IUrlParameter.GetString`. - -[discrete] -==== Remove Metric.GetString(ITransportConfiguration settings) method - -This method is used internally by the client and should not be exposed to -consuming applications. Instead, we prefer explicit interface implementation for -`IUrlParameter.GetString`. - -[discrete] -==== Remove TimeStamp.GetString(ITransportConfiguration settings) method - -This method is used internally by the client and should not be exposed to -consuming applications. Instead, we prefer explicit interface implementation for -`IUrlParameter.GetString`. - -[discrete] -==== Remove IndexUuid.GetString(ITransportConfiguration settings) method - -This method is used internally by the client and should not be exposed to -consuming applications. Instead, we prefer explicit interface implementation for -`IUrlParameter.GetString`. - -[discrete] -==== Remove TaskId.GetString(ITransportConfiguration settings) method - -This method is used internally by the client and should not be exposed to -consuming applications. Instead, we prefer explicit interface implementation for -`IUrlParameter.GetString`. - -[discrete] -==== The Metric type is now sealed - -This type has been sealed to align with other types for consistency. We don’t -expect consumers to derive from this type. \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.0.3.asciidoc b/docs/release-notes/release-notes-8.0.3.asciidoc deleted file mode 100644 index 64c8dccfedf..00000000000 --- a/docs/release-notes/release-notes-8.0.3.asciidoc +++ /dev/null @@ -1,17 +0,0 @@ -[[release-notes-8.0.3]] -== Release notes v8.0.3 - -[discrete] -=== Bug fixes - -- Fix field sort serialization (issue: -https://github.com/elastic/elasticsearch-net/issues/7074[#7074]) - -[discrete] -=== Enhancements - -[discrete] -==== Update to Elastic.Transport 0.4.5 - -Upgrades the client to depend on the 0.4.5 release of Elastic.Transport which -includes automatic sending of https://www.elastic.co/guide/en/elasticsearch/reference/current/rest-api-compatibility.html#rest-api-compatibility[REST API compatibility] headers for Elasticsearch requests. \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.0.4.asciidoc b/docs/release-notes/release-notes-8.0.4.asciidoc deleted file mode 100644 index ac61771ebde..00000000000 --- a/docs/release-notes/release-notes-8.0.4.asciidoc +++ /dev/null @@ -1,138 +0,0 @@ -[[release-notes-8.0.4]] -== Release notes v8.0.4 - -[discrete] -=== Bug fixes - -- Fix code-gen for IndexSettingsAnalysis (issue: -https://github.com/elastic/elasticsearch-net/issues/7118[#7118]) -- Complete implementation of Metrics type -- Update generated code with fixes from 8.6 specification (issue: -https://github.com/elastic/elasticsearch-net/issues/7119[#7119]). Adds `Missing` -property to `MultiTermLookup`. - -[discrete] -=== Breaking changes - -In the course of fixing the code-generation of types used on `IndexSettingsAnalysis`, -several breaking changes were introduced. Some of these were necessary to make the -types usable, while others fixed the consistency of the generated code. - -[discrete] -==== IndexSettingsAnalysis - -Code-generation has been updated to apply transforms to fix the specification -of the `IndexSettingsAnalysis` type. As a result, all properties have been renamed, -and some property types have been changed. - -* The `Analyzer` property is now pluralized and renamed to `Analyzers` to align with -NEST and make it clearer that this can contain more than one analyzer definition. -* The `CharFilter` property is now pluralized and renamed to `CharFilters` to align with -NEST and make it clearer that this can contain more than one char filter definition. -Its type has changes from a `IDictionary` -to `CharFilters`, a tagged union type deriving from IsADictionary`. -* The `Filter` property is now pluralized and renamed to `TokenFilters` to align with -NEST and make it clearer that this can contain more than one token filter definition. -Its type has changes from a `IDictionary` -to `TokenFilters`, a tagged union type deriving from IsADictionary`. -* The `Normalizer` property is now pluralized and renamed to `Normalizers` to align with -NEST and make it clearer that this can contain more than one normalizer definition. -* The `Tokenizer` property is now pluralized and renamed to `Tokenizers` to align with -NEST and make it clearer that this can contain more than one tokenizer definition. -Its type has changes from a `IDictionary` -to `TokenFilters`, a tagged union type deriving from IsADictionary`. - -*_Before_* - -[source,csharp] ----- -public sealed partial class IndexSettingsAnalysis -{ - public Elastic.Clients.Elasticsearch.Analysis.Analyzers? Analyzer { get; set; } - public IDictionary? CharFilter { get; set; } - public IDictionary? Filter { get; set; } - public Elastic.Clients.Elasticsearch.Analysis.Normalizers? Normalizer { get; set; } - public IDictionary? Tokenizer { get; set; } -} ----- - -*_After_* - -[source,csharp] ----- -public sealed partial class IndexSettingsAnalysis -{ - public Elastic.Clients.Elasticsearch.Analysis.Analyzers? Analyzers { get; set; } - public Elastic.Clients.Elasticsearch.Analysis.CharFilters? CharFilters { get; set; } - public Elastic.Clients.Elasticsearch.Analysis.TokenFilters? TokenFilters { get; set; } - public Elastic.Clients.Elasticsearch.Analysis.Normalizers? Normalizers { get; set; } - public Elastic.Clients.Elasticsearch.Analysis.Tokenizers? Tokenizers { get; set; } -} ----- - -The `IndexSettingsAnalysisDescriptor` type has been updated accordingly to apply -the above changes. It now supports a more convenient syntax to easily define -the filters, normalizers and tokenizers that apply to the settings for indices. - -[discrete] -===== Example usage of updated fluent syntax: - -[source,csharp] ----- -var descriptor = new CreateIndexRequestDescriptor("test") - .Settings(s => s - .Analysis(a => a - .Analyzers(a => a - .Stop("stop-name", stop => stop.StopwordsPath("analysis/path.txt")) - .Pattern("pattern-name", pattern => pattern.Version("version")) - .Custom("my-custom-analyzer", c => c - .Filter(new[] { "stop", "synonym" }) - .Tokenizer("standard"))) - .TokenFilters(f => f - .Synonym("synonym", synonym => synonym - .SynonymsPath("analysis/synonym.txt"))))); ----- - -[discrete] -==== Token Filters - -Token filter types now implement the `ITokenFilter` interface, rather than -`ITokenFilterDefinition`. - -The `TokenFilter` union type has been renamed to `CategorizationTokenFilter` to -clearly signify it's use only within ML categorization contexts. - -A `TokenFilters` type has been introduced, which derives from `IsADictionary` and -supports convenient addition of known token filters via the fluent API. - -[discrete] -==== Character Filters - -Character filter types now implement the `ICharFilter` interface, rather than -`ICharFilterDefinition`. - -The `CharFilter` union type has been renamed to `CategorizationCharFilter` to -clearly signify it's use only within ML categorization contexts. - -A `CharFilters` type has been introduced, which derives from `IsADictionary` and -supports convenient addition of known character filters via the fluent API. - -[discrete] -==== Tokenizers - -Tokenizer types now implement the `ITokenizer` interface, rather than -`ITokenizerDefinition`. - -The `Tokenizer` union type has been renamed to `CategorizationTokenizer` to -clearly signify it's use only within ML categorization contexts. - -A `Tokenizers` type has been introduced, which derives from `IsADictionary` and -supports convenient addition of known tokenizers via the fluent API. - -[discrete] -==== IndexManagement.StorageType - -The 8.6 specification fixed this type to mark is as a non-exhaustive enum, since -it supports additional values besides those coded into the specification. As a -result the code-generation for this type causes some breaking changes. The type -is no longer generated as an `enum` and is not a custom `readonly struct`. \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.0.5.asciidoc b/docs/release-notes/release-notes-8.0.5.asciidoc deleted file mode 100644 index 15961356b15..00000000000 --- a/docs/release-notes/release-notes-8.0.5.asciidoc +++ /dev/null @@ -1,98 +0,0 @@ -[[release-notes-8.0.5]] -== Release notes v8.0.5 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7171[#7171] Fix code-gen for IndexTemplate (issue: https://github.com/elastic/elasticsearch-net/issues/7161[#7161]) -- https://github.com/elastic/elasticsearch-net/pull/7181[#7181] Fix MultiGet response deserialization for non-matched IDs (issue: https://github.com/elastic/elasticsearch-net/issues/7169[#7169]) -- https://github.com/elastic/elasticsearch-net/pull/7182[#7182] Implement Write method on SourceConfigConverter (issue: https://github.com/elastic/elasticsearch-net/issues/7170[#7170]) -- https://github.com/elastic/elasticsearch-net/pull/7205[#7205] Update to Elastic.Transport to 0.4.6 which improves the version detection used by the REST API compatibility Accept header - -[discrete] -=== Breaking changes - -In the course of fixing the code-generation for index templates to avoid serialization failures, some breaking changes were introduced. - -[discrete] -==== IndexTemplate - -`IndexTemplate` forms part of the `IndexTemplateItem` included on `GetIndexTemplateResponse`. - -* The type for the `ComposedOf` property has changed from `IReadOnlyCollection` to `IReadOnlyCollection` -* The type for the `IndexPatterns` property has changed from `Elastic.Clients.Elasticsearch.Names` to `IReadOnlyCollection` - -*_Before_* - -[source,csharp] ----- -public sealed partial class IndexTemplate -{ - ... - public IReadOnlyCollection ComposedOf { get; init; } - public Elastic.Clients.Elasticsearch.Names IndexPatterns { get; init; } - ... -} ----- - -*_After_* - -[source,csharp] ----- -public sealed partial class IndexTemplate -{ - ... - public IReadOnlyCollection ComposedOf { get; init; } - public IReadOnlyCollection IndexPatterns { get; init; } - ... -} ----- - -[discrete] -==== SimulateIndexTemplateRequest - -* The type for the `ComposedOf` property has changed from `IReadOnlyCollection` to `IReadOnlyCollection` - -*_Before_* - -[source,csharp] ----- -public sealed partial class SimulateIndexTemplateRequest -{ - ... - public IReadOnlyCollection? ComposedOf { get; set; } - ... -} ----- - -*_After_* - -[source,csharp] ----- -public sealed partial class SimulateIndexTemplateRequest -{ - ... - public IReadOnlyCollection? ComposedOf { get; set; } - ... -} ----- - -[discrete] -==== SimulateIndexTemplateRequestDescriptor and SimulateIndexTemplateRequestDescriptor - -The `ComposedOf` method signature has changed to accept a parameter of `ICollection?` instead of -`ICollection?`. - -*_Before_* - -[source,csharp] ----- -public SimulateIndexTemplateRequestDescriptor ComposedOf(ICollection? composedOf) ----- - -*_After_* - -[source,csharp] ----- -public SimulateIndexTemplateRequestDescriptor ComposedOf(ICollection? composedOf) ----- \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.0.6.asciidoc b/docs/release-notes/release-notes-8.0.6.asciidoc deleted file mode 100644 index 301f18470fd..00000000000 --- a/docs/release-notes/release-notes-8.0.6.asciidoc +++ /dev/null @@ -1,110 +0,0 @@ -[[release-notes-8.0.6]] -== Release notes v8.0.6 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7244[#7244] Fix code-gen for -single or many types. Includes support for deserializing numbers represented as -strings in the JSON payload. (issues: https://github.com/elastic/elasticsearch-net/issues/7221[#7221], -https://github.com/elastic/elasticsearch-net/issues/7234[#7234], -https://github.com/elastic/elasticsearch-net/issues/7240[#7240]). -- https://github.com/elastic/elasticsearch-net/pull/7253[#7253] Fix code-gen for -enums with aliases (issue: https://github.com/elastic/elasticsearch-net/issues/7236[#7236]) -- https://github.com/elastic/elasticsearch-net/pull/7262[#7262] Update to -`Elastic.Transport` 0.4.7 which includes fixes for helpers used during application -testing. - -[discrete] -=== Features - -- https://github.com/elastic/elasticsearch-net/pull/7272[#7272] Support custom JsonSerializerOptions. - -[discrete] -=== Breaking changes - -[discrete] -==== DynamicTemplate - -`DynamicTemplate` forms part of the `TypeMapping` object, included on `GetIndexRespone`. - -* The type for the `Mapping` property has changed from `Elastic.Clients.Elasticsearch.Properties` -to `Elastic.Clients.Elasticsearch.IProperty`. This breaking change fixes an error -introduced by the code-generator. Before introducing this fix, the type could -not correctly deserialize responses for GET index requests and prevented dynamic -templates from being configured for indices via PUT index. - -*_Before_* - -[source,csharp] ----- -public sealed partial class DynamicTemplate -{ - ... - public Elastic.Clients.Elasticsearch.Mapping.Properties? Mapping { get; set; } - ... -} ----- - -*_After_* - -[source,csharp] ----- -public sealed partial class DynamicTemplate -{ - ... - public Elastic.Clients.Elasticsearch.Mapping.IProperty? Mapping { get; set; } - ... -} ----- - -[discrete] -==== TypeMapping - -Among other uses, `TypeMapping` forms part of the `GetIndexRespone`. - -* The `DynamicTemplates` property has been simplified to make it easier to work -with and to fix deserialization failures on certain responses. Rather than use a -`Union` to describe the fact that this property may be a single dictionary of -dynamic templates, or an array of dictionaries, this is now code-generated as a -specialised single or many collection. The API exposes this as an `ICollection` -of dictionaries and the JSON converter is able to handle either an array or -individual dictionary in responses. - -*_Before_* - -[source,csharp] ----- -public sealed partial class TypeMapping -{ - ... - public Union?, ICollection>?>? DynamicTemplates { get; set; } - ... -} ----- - -*_After_* - -[source,csharp] ----- -public sealed partial class TypeMapping -{ - ... - public ICollection>? DynamicTemplates { get; set; } - ... -} ----- - -[discrete] -==== SystemTextJsonSerializer - -The `SystemTextJsonSerializer` is used as a base type for the built-in serializers. Two breaking changes have been made after adding better support for <>. - -The public `Options` property has been made internal. - -A new public abstract method `CreateJsonSerializerOptions` has been added, which derived types must implement. - -[source,csharp] ----- -protected abstract JsonSerializerOptions CreateJsonSerializerOptions(); ----- diff --git a/docs/release-notes/release-notes-8.0.7.asciidoc b/docs/release-notes/release-notes-8.0.7.asciidoc deleted file mode 100644 index fd5c2261708..00000000000 --- a/docs/release-notes/release-notes-8.0.7.asciidoc +++ /dev/null @@ -1,7 +0,0 @@ -[[release-notes-8.0.7]] -== Release notes v8.0.7 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7337[#7337] Fix code-gen for dynamic_templates. (issue: https://github.com/elastic/elasticsearch-net/issues/7234[#7234]) \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.0.8.asciidoc b/docs/release-notes/release-notes-8.0.8.asciidoc deleted file mode 100644 index 9952e1c6cee..00000000000 --- a/docs/release-notes/release-notes-8.0.8.asciidoc +++ /dev/null @@ -1,7 +0,0 @@ -[[release-notes-8.0.8]] -== Release notes v8.0.8 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7456[#7456] Fix CompletionSuggester based on spec fixes. (issue: https://github.com/elastic/elasticsearch-net/issues/7454[#7454]) diff --git a/docs/release-notes/release-notes-8.0.9.asciidoc b/docs/release-notes/release-notes-8.0.9.asciidoc deleted file mode 100644 index b9086a404b5..00000000000 --- a/docs/release-notes/release-notes-8.0.9.asciidoc +++ /dev/null @@ -1,34 +0,0 @@ -[[release-notes-8.0.9]] -== Release notes v8.0.9 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7446[#7446] Fix byte properties -in index stats types. (issue: https://github.com/elastic/elasticsearch-net/issues/7445[#7445]) - -[discrete] -=== Enhancements - -- https://github.com/elastic/elasticsearch-net/pull/7467[#7467] Source serialization -always sends fractional format for double and floats. (issue: https://github.com/elastic/elasticsearch-net/issues/7051[#7051]) - -[discrete] -=== Breaking changes - -[discrete] -==== Source serialization of float and double properties - -By default, when serializing `double` and `float` properties, the `System.Text.Json` -serializer uses the "G17" format when serializing double types. This format omits -the decimal point and/or trailing zeros if they are not required for the data to -roundtrip. This is generally correct, as JSON doesn't specify a type for numbers. - -However, in the case of source serialization, mappings for numeric properties may -be incorrectly inferred if trailing zeros are omitted. In this release, we have -included a new custom converter for `float` and `double` types when serialized using -the default source serializer. These converters ensure that at least one precision -digit is included after a decimal point, even for round numbers. - -You may therefore observe changes to the serialized source document after -upgrading to this version. \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.1.0.asciidoc b/docs/release-notes/release-notes-8.1.0.asciidoc deleted file mode 100644 index 3fdac1643ff..00000000000 --- a/docs/release-notes/release-notes-8.1.0.asciidoc +++ /dev/null @@ -1,90 +0,0 @@ -[[release-notes-8.1.0]] -== Release notes v8.1.0 - -A core theme of the 8.1.0 release is the reintroduction of many features which -were missing from the 8.0 releases. The 8.x client still does NOT have full -feature parity with NEST and we continue to work on closing these gaps. - -[discrete] -=== Enhancements - -[discrete] -==== Support for additional endpoints - -Adds support for the following endpoints: - -- Cluster.AllocationExplain -- Cluster.Stats -- Cluster.PendingTasks -- DanglingIndices.List -- Enrich.DeletePolicy -- Enrich.ExecutePolicy -- Enrich.PutPolicy -- Enrich.Stats -- Graph.Explore -- IndexManagement.UpdateAliases -- Ingest.GeoIpStats -- Ingest.GetPipeline -- Ingest.ProcessorGrok -- Ingest.PutPipeline -- Ingest.Simulate -- MultiTermVectors -- RenderSearchTemplate -- SearchTemplate -- Tasks.Cancel -- Tasks.Get -- Tasks.List -- TermVectors - -[discrete] -==== Support for additional queries - -Adds support for the following queries: - -- Geo distance -- Geo bounding box -- Geo polygon -- Pinned -- Range queries (date and numeric) -- Raw (can be used as a client specific fallback for missing queries by sending raw JSON) - -[discrete] -==== Support for additional aggregations - -Adds support for the following aggregations: - -- Boxplot -- Bucket sort -- Composite -- Cumulative sum -- Geo bounds -- Geo centroid -- Geo distance -- Geo line -- Geohash grid -- Geohex grid -- Geotile grid -- IP prefix -- Multi terms -- Rare terms -- Significant terms -- Weighted average - -[discrete] -==== Other enhancements - -- *Add support for geo distance sorting.* -Adds support for specifying a `GeoDistanceSort` on `SortOptions`. -- *Add support for weight score on FunctionScore.* -Adds support for specifying a weight score value on the `FunctionScore` type. -- *Code generate XML doc comments.* -The code generator now adds XML doc comments to types and members when present in -the Elasticsearch specification. This acts as an aid when exploring the API in an -IDE such as Visual Studio. -- *Add additional client overloads.* -Adds additional overloads to the `ElasticsearchClient` and namespaced sub-clients -that allow consumers to provide a descriptor instance used when building requests. -- *Add support for bool query operators in Query DSL for object initializer syntax* -Adds support for using operators `&&``, `||`, `!` and `+` to build up bool queries -using the object initializer syntax. NOTE: Operators are not yet supported for -combining queires defined using the fluent descriptor syntax. \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.1.1.asciidoc b/docs/release-notes/release-notes-8.1.1.asciidoc deleted file mode 100644 index 100b6e99f12..00000000000 --- a/docs/release-notes/release-notes-8.1.1.asciidoc +++ /dev/null @@ -1,72 +0,0 @@ -[[release-notes-8.1.1]] -== Release notes v8.1.1 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7667[#7667] Fix SQL missing -Rows on QueryResponse (issue: https://github.com/elastic/elasticsearch-net/issues/7663[#7663]) -- https://github.com/elastic/elasticsearch-net/pull/7676[#7676] Ensure async client -methods pass through cancellation token (issue: https://github.com/elastic/elasticsearch-net/issues/7665[#7665]) - -[discrete] -=== Enhancements - -- https://github.com/elastic/elasticsearch-net/pull/7684[#7684] Regenerated code -with latest spec fixes for 8.7 - -[discrete] -=== Breaking changes - -This release includes the following breaking changes as a result of specification fixes: - -[discrete] -==== AsyncSearch and MultisearchBody KnnQuery - -The type for the `SubmitAsyncSearchRequest.Knn` and `MultisearchBody.Knn` properties -has changed to an `ICollection` from a single `KnnQuery` since it is -possible to include more than one query in a request. - -*_Before_* - -[source,csharp] ----- -public sealed partial class SubmitAsyncSearchRequest -{ - ... - public Elastic.Clients.Elasticsearch.KnnQuery? Knn { get; set; } - ... -} ----- - -[source,csharp] ----- -public sealed partial class MultisearchBody -{ - ... - public Elastic.Clients.Elasticsearch.KnnQuery? Knn { get; set; } - ... -} ----- - -*_After_* - -[source,csharp] ----- -public sealed partial class SubmitAsyncSearchRequest -{ - ... - public ICollection? Knn { get; set; } - ... -} ----- - -[source,csharp] ----- -public sealed partial class MultisearchBody -{ - ... - public ICollection? Knn { get; set; } - ... -} ----- \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.1.2.asciidoc b/docs/release-notes/release-notes-8.1.2.asciidoc deleted file mode 100644 index 71c34bca8b2..00000000000 --- a/docs/release-notes/release-notes-8.1.2.asciidoc +++ /dev/null @@ -1,17 +0,0 @@ -[[release-notes-8.1.2]] -== Release notes v8.1.2 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7718[#7718] Regen index setting blocks based on fixed spec (issue: https://github.com/elastic/elasticsearch-net/issues/7714[#7714]) - -[discrete] -=== Enhancements - -- https://github.com/elastic/elasticsearch-net/pull/7781[#7781] Bump dependencies (issue: https://github.com/elastic/elasticsearch-net/issues/7752[#7752]) - -[discrete] -=== Docs - -- https://github.com/elastic/elasticsearch-net/pull/7772[#7772] [Backport 8.1] [Backport 8.7][DOCS] Adds getting started content based on the template (issue: https://github.com/elastic/elasticsearch-net/pull/7770[#7770]) \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.1.3.asciidoc b/docs/release-notes/release-notes-8.1.3.asciidoc deleted file mode 100644 index e436f226717..00000000000 --- a/docs/release-notes/release-notes-8.1.3.asciidoc +++ /dev/null @@ -1,19 +0,0 @@ -[[release-notes-8.1.3]] -== Release notes v8.1.3 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7737[#7737] Boosted non-exhaustive enum deserialization (issue: https://github.com/elastic/elasticsearch-net/issues/7729[#7729]) -- https://github.com/elastic/elasticsearch-net/pull/7738[#7738] Complted buckets JSON converter (issue: https://github.com/elastic/elasticsearch-net/issues/7713[#7713]) -- https://github.com/elastic/elasticsearch-net/pull/7753[#7753] Number converters should not fall through and throw exceptions in non NETCore builds (issue: https://github.com/elastic/elasticsearch-net/issues/7757[#7757]) -- https://github.com/elastic/elasticsearch-net/pull/7811[#7811] Fix localization issue with floating-point deserialization from string - -[discrete] -=== Enhancements - -- https://github.com/elastic/elasticsearch-net/pull/7730[#7730] Refactoring and tiny behavior fix for Ids -- https://github.com/elastic/elasticsearch-net/pull/7731[#7731] No allocations in `ResponseItem.IsValid`` property -- https://github.com/elastic/elasticsearch-net/pull/7733[#7733] Fixed the equality contract on Metrics type -- https://github.com/elastic/elasticsearch-net/pull/7735[#7735] Removed unused `JsonIgnore` -- https://github.com/elastic/elasticsearch-net/pull/7736[#7736] Optimized `FieldConverter` \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.10.0.asciidoc b/docs/release-notes/release-notes-8.10.0.asciidoc deleted file mode 100644 index 9b587de7bea..00000000000 --- a/docs/release-notes/release-notes-8.10.0.asciidoc +++ /dev/null @@ -1,13 +0,0 @@ -[[release-notes-8.10.0]] -== Release notes v8.10.0 - -[discrete] -=== Features & Enhancements - -- https://github.com/elastic/elasticsearch-net/pull/7931[#7931] Refactor OpenTelemetry implementation with updated Transport (issue: https://github.com/elastic/elasticsearch-net/issues/7885[#7885]) -- https://github.com/elastic/elasticsearch-net/pull/7953[#7953] Add `TDigestPercentilesAggregate` (issues: https://github.com/elastic/elasticsearch-net/issues/7923[#7923], https://github.com/elastic/elasticsearch-net/issues/7879[#7879]) - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7956[#7956] Add `Similarity` to `KnnQuery` (issue: https://github.com/elastic/elasticsearch-net/issues/7952[#7952]) \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.11.0.asciidoc b/docs/release-notes/release-notes-8.11.0.asciidoc deleted file mode 100644 index 1e002cb0b4e..00000000000 --- a/docs/release-notes/release-notes-8.11.0.asciidoc +++ /dev/null @@ -1,13 +0,0 @@ -[[release-notes-8.11.0]] -== Release notes v8.11.0 - -[discrete] -=== Features & Enhancements - -- https://github.com/elastic/elasticsearch-net/pull/7978[#7978] Regenerate client for 8.11 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7979[#7979] Add workaround for stringified properties which are not marked properly in specification -- https://github.com/elastic/elasticsearch-net/pull/7965[#7965] Fix `Stringified` converters \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.9.0.asciidoc b/docs/release-notes/release-notes-8.9.0.asciidoc deleted file mode 100644 index 0a622988037..00000000000 --- a/docs/release-notes/release-notes-8.9.0.asciidoc +++ /dev/null @@ -1,15 +0,0 @@ -[[release-notes-8.9.0]] -== Release notes v8.9.0 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7839[#7839] Use `Stringified` for `preserve_original` and `indexing_complete` (issue: https://github.com/elastic/elasticsearch-net/issues/7755[#7755]) -- https://github.com/elastic/elasticsearch-net/pull/7840[#7840] Update `Elastic.*` dependencies (issue: https://github.com/elastic/elasticsearch-net/issues/7823[#7823]) -- https://github.com/elastic/elasticsearch-net/pull/7841[#7841] Fix typing of `BulkUpdateOperation.RetryOnConflict` (issue: https://github.com/elastic/elasticsearch-net/issues/7838[#7838]) -- https://github.com/elastic/elasticsearch-net/pull/7854[#7854] Fix custom floating-point JSON converters (issue: https://github.com/elastic/elasticsearch-net/issues/7757[#7757]) - -[discrete] -=== Enhancements - -- https://github.com/elastic/elasticsearch-net/pull/7836[#7836] Regenerate client using 8.9 specification \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.9.1.asciidoc b/docs/release-notes/release-notes-8.9.1.asciidoc deleted file mode 100644 index ea5a4f19aeb..00000000000 --- a/docs/release-notes/release-notes-8.9.1.asciidoc +++ /dev/null @@ -1,7 +0,0 @@ -[[release-notes-8.9.1]] -== Release notes v8.9.1 - -[discrete] -=== Bug fixes - -- https://github.com/elastic/elasticsearch-net/pull/7864[#7864] Fix `TextExpansionQuery` definition diff --git a/docs/release-notes/release-notes-8.9.2.asciidoc b/docs/release-notes/release-notes-8.9.2.asciidoc deleted file mode 100644 index e148e5a1673..00000000000 --- a/docs/release-notes/release-notes-8.9.2.asciidoc +++ /dev/null @@ -1,14 +0,0 @@ -[[release-notes-8.9.2]] -== Release notes v8.9.2 - -[discrete] -=== Bug fixes - - - https://github.com/elastic/elasticsearch-net/pull/7875[#7875] Fix `aggregations` property not being generated for `MultisearchBody` (issue https://github.com/elastic/elasticsearch-net/issues/7873[#7873]) - - https://github.com/elastic/elasticsearch-net/pull/7875[#7875] Remove invalid properties from `SlowlogTresholds` (issue https://github.com/elastic/elasticsearch-net/issues/7865[#7865]) - - https://github.com/elastic/elasticsearch-net/pull/7883[#7883] Remove leading `/` character from API urls (issue: https://github.com/elastic/elasticsearch-net/issues/7878[#7878]) - -[discrete] -=== Features & Enhancements - -- https://github.com/elastic/elasticsearch-net/pull/7869[#7869] Add support for `SimpleQueryStringQuery.flags property (issue: https://github.com/elastic/elasticsearch-net/issues/7863[#7863]) \ No newline at end of file diff --git a/docs/release-notes/release-notes-8.9.3.asciidoc b/docs/release-notes/release-notes-8.9.3.asciidoc deleted file mode 100644 index efffe9685d9..00000000000 --- a/docs/release-notes/release-notes-8.9.3.asciidoc +++ /dev/null @@ -1,10 +0,0 @@ -[[release-notes-8.9.3]] -== Release notes v8.9.3 - -[discrete] -=== Features & Enhancements - -- https://github.com/elastic/elasticsearch-net/pull/7894[#7894] Reintroduce suggestion feature (issue: https://github.com/elastic/elasticsearch-net/issues/7390[#7390]) -- https://github.com/elastic/elasticsearch-net/pull/7923[#7923] Add `PercentilesAggregation` and `PercentileRanksAggregation` (issue: https://github.com/elastic/elasticsearch-net/issues/7879[#7879]) -- https://github.com/elastic/elasticsearch-net/pull/7914[#7914] Update `Elastic.Transport` dependency -- https://github.com/elastic/elasticsearch-net/pull/7920[#7920] Regenerate client using the latest specification \ No newline at end of file diff --git a/docs/release-notes/release-notes.asciidoc b/docs/release-notes/release-notes.asciidoc deleted file mode 100644 index 71416f69ca9..00000000000 --- a/docs/release-notes/release-notes.asciidoc +++ /dev/null @@ -1,68 +0,0 @@ -[[release-notes]] -= Release notes - -* <> - -[discrete] -== Version 8.11 - -* <> - -[discrete] -== Version 8.10 - -* <> - -[discrete] -== Version 8.9 - -* <> -* <> -* <> -* <> - -[discrete] -== Version 8.1 - -* <> -* <> -* <> -* <> - -[discrete] -== Version 8.0 - -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> - -include::breaking-change-policy.asciidoc[] -include::release-notes-8.11.0.asciidoc[] -include::release-notes-8.10.0.asciidoc[] -include::release-notes-8.9.3.asciidoc[] -include::release-notes-8.9.2.asciidoc[] -include::release-notes-8.9.1.asciidoc[] -include::release-notes-8.9.0.asciidoc[] -include::release-notes-8.1.3.asciidoc[] -include::release-notes-8.1.2.asciidoc[] -include::release-notes-8.1.1.asciidoc[] -include::release-notes-8.1.0.asciidoc[] -include::release-notes-8.0.10.asciidoc[] -include::release-notes-8.0.9.asciidoc[] -include::release-notes-8.0.8.asciidoc[] -include::release-notes-8.0.7.asciidoc[] -include::release-notes-8.0.6.asciidoc[] -include::release-notes-8.0.5.asciidoc[] -include::release-notes-8.0.4.asciidoc[] -include::release-notes-8.0.3.asciidoc[] -include::release-notes-8.0.2.asciidoc[] -include::release-notes-8.0.1.asciidoc[] -include::release-notes-8.0.0.asciidoc[] \ No newline at end of file diff --git a/docs/release-notes/toc.yml b/docs/release-notes/toc.yml new file mode 100644 index 00000000000..a4100679473 --- /dev/null +++ b/docs/release-notes/toc.yml @@ -0,0 +1,5 @@ +toc: + - file: index.md + - file: known-issues.md + - file: breaking-changes.md + - file: deprecations.md \ No newline at end of file diff --git a/docs/troubleshooting.asciidoc b/docs/troubleshooting.asciidoc deleted file mode 100644 index c30c6bac554..00000000000 --- a/docs/troubleshooting.asciidoc +++ /dev/null @@ -1,45 +0,0 @@ -[[troubleshooting]] -= Troubleshooting - -[partintro] --- -The client can provide rich details about what occurred in the request pipeline during the process -of making a request, as well as be configured to provide the raw request and response JSON - -* <> - -* <> - --- - -[[logging]] -== Logging - -Whilst developing with Elasticsearch using NEST, it can be extremely valuable to see the requests that -NEST generates and sends to Elasticsearch, as well as the responses returned. - -There are a couple of popular ways of capturing this information - -* <> - -* <> - -include::client-concepts/troubleshooting/logging-with-on-request-completed.asciidoc[] - -include::client-concepts/troubleshooting/logging-with-fiddler.asciidoc[] - -[[debugging]] -== Debugging - -When things are going awry, you want to be provided with as much information as possible, to resolve -the issue! - -Elasticsearch.Net and NEST provide an <> and <> to -help get you back on the happy path. - -include::client-concepts/troubleshooting/audit-trail.asciidoc[] - -include::client-concepts/troubleshooting/debug-information.asciidoc[] - -include::client-concepts/troubleshooting/debug-mode.asciidoc[] - diff --git a/docs/usage/esql.asciidoc b/docs/usage/esql.asciidoc deleted file mode 100644 index 7b7c1a0fe42..00000000000 --- a/docs/usage/esql.asciidoc +++ /dev/null @@ -1,69 +0,0 @@ -[[esql]] -== ES|QL in the .NET client -++++ -Using ES|QL -++++ - -This page helps you understand and use {ref}/esql.html[ES|QL] in the -.NET client. - -There are two ways to use ES|QL in the .NET client: - -* Use the Elasticsearch {es-docs}/esql-apis.html[ES|QL API] directly: This -is the most flexible approach, but it's also the most complex because you must handle -results in their raw form. You can choose the precise format of results, -such as JSON, CSV, or text. -* Use ES|QL high-level helpers: These helpers take care of parsing the raw -response into something readily usable by the application. Several helpers are -available for different use cases, such as object mapping, cursor -traversal of results (in development), and dataframes (in development). - -[discrete] -[[esql-how-to]] -=== How to use the ES|QL API - -The {es-docs}/esql-query-api.html[ES|QL query API] allows you to specify how -results should be returned. You can choose a -{es-docs}/esql-rest.html#esql-rest-format[response format] such as CSV, text, or -JSON, then fine-tune it with parameters like column separators -and locale. - -The following example gets ES|QL results as CSV and parses them: - -[source,charp] ----- -var response = await client.Esql.QueryAsync(r => r - .Query("FROM index") - .Format("csv") -); -var csvContents = Encoding.UTF8.GetString(response.Data); ----- - -[discrete] -[[esql-consume-results]] -=== Consume ES|QL results - -The previous example showed that although the raw ES|QL API offers maximum -flexibility, additional work is required in order to make use of the -result data. - -To simplify things, try working with these three main representations of ES|QL -results (each with its own mapping helper): - -* **Objects**, where each row in the results is mapped to an object from your -application domain. This is similar to what ORMs (object relational mappers) -commonly do. -* **Cursors**, where you scan the results row by row and access the data using -column names. This is similar to database access libraries. -* **Dataframes**, where results are organized in a column-oriented structure that -allows efficient processing of column data. - -[source,charp] ----- -// ObjectAPI example -var response = await client.Esql.QueryAsObjectsAsync(x => x.Query("FROM index")); -foreach (var person in response) -{ - // ... -} ----- diff --git a/docs/usage/examples.asciidoc b/docs/usage/examples.asciidoc deleted file mode 100644 index 501c63b89e9..00000000000 --- a/docs/usage/examples.asciidoc +++ /dev/null @@ -1,122 +0,0 @@ -[[examples]] -== CRUD usage examples - -This page helps you to understand how to perform various basic {es} CRUD -(create, read, update, delete) operations using the .NET client. It demonstrates -how to create a document by indexing an object into {es}, read a document back, -retrieving it by ID or performing a search, update one of the fields in a -document and delete a specific document. - -These examples assume you have an instance of the `ElasticsearchClient` -accessible via a local variable named `client` and several using directives in -your C# file. - -[source,csharp] ----- -include::{doc-tests-src}/Usage/CrudExamplesTests.cs[tags=using-directives;create-client] ----- -<1> The default constructor, assumes an unsecured {es} server is running and -exposed on 'http://localhost:9200'. See <> for examples -of connecting to secured servers and https://www.elastic.co/cloud[Elastic Cloud] -deployments. - -The examples operate on data representing tweets. Tweets are modelled in the -client application using a C# class named 'Tweet' containing several properties -that map to the document structure being stored in {es}. - -[source,csharp] ----- -include::{doc-tests-src}/Usage/CrudExamplesTests.cs[tag=tweet-class] ----- -<1> By default, the .NET client will try to find a property called `Id` on the -class. When such a property is present it will index the document into {es} -using the ID specified by the value of this property. - - -[discrete] -[[indexing-net]] -=== Indexing a document - -Documents can be indexed by creating an instance representing a tweet and -indexing it via the client. In these examples, we will work with an index named -'my-tweet-index'. - -[source,csharp] ----- -include::{doc-tests-src}/Usage/CrudExamplesTests.cs[tag=create-tweet] ----- -<1> Create an instance of the `Tweet` class with relevant properties set. -<2> Prefer the async APIs, which require awaiting the response. -<3> Check the `IsValid` property on the response to confirm that the request and -operation succeeded. -<4> Access the `IndexResponse` properties, such as the ID, if necessary. - -[discrete] -[[getting-net]] -=== Getting a document - -[source,csharp] ----- -include::{doc-tests-src}/Usage/CrudExamplesTests.cs[tag=get-tweet] ----- -<1> The `GetResponse` is mapped 1-to-1 with the Elasticsearch JSON response. -<2> The original document is deserialized as an instance of the Tweet class, -accessible on the response via the `Source` property. - - -[discrete] -[[searching-net]] -=== Searching for documents - -The client exposes a fluent interface and a powerful query DSL for searching. - -[source,csharp] ----- -include::{doc-tests-src}/Usage/CrudExamplesTests.cs[tag=search-tweet-fluent] ----- -<1> The generic type argument specifies the `Tweet` class, which is used when -deserialising the hits from the response. -<2> The index can be omitted if a `DefaultIndex` has been configured on -`ElasticsearchClientSettings`, or a specific index was configured when mapping -this type. -<3> Execute a term query against the `user` field, searching for tweets authored -by the user 'stevejgordon'. -<4> Documents matched by the query are accessible via the `Documents` collection -property on the `SearchResponse`. - -You may prefer using the object initializer syntax for requests if lambdas -aren't your thing. - -[source,csharp] ----- -include::{doc-tests-src}/Usage/CrudExamplesTests.cs[tag=search-tweet-object-initializer] ----- -<1> Create an instance of `SearchRequest`, setting properties to control the -search operation. -<2> Pass the request to the `SearchAsync` method on the client. - -[discrete] -[[updating-net]] -=== Updating documents - -Documents can be updated in several ways, including by providing a complete -replacement for an existing document ID. - -[source,csharp] ----- -include::{doc-tests-src}/Usage/CrudExamplesTests.cs[tag=update-tweet] ----- -<1> Update a property on the existing tweet instance. -<2> Send the updated tweet object in the update request. - - -[discrete] -[[deleting-net]] -=== Deleting documents - -Documents can be deleted by providing the ID of the document to remove. - -[source,csharp] ----- -include::{doc-tests-src}/Usage/CrudExamplesTests.cs[tag=delete-tweet] ----- diff --git a/docs/usage/index.asciidoc b/docs/usage/index.asciidoc deleted file mode 100644 index f4ae4474730..00000000000 --- a/docs/usage/index.asciidoc +++ /dev/null @@ -1,25 +0,0 @@ -[[usage]] -= Using the .NET Client - -[partintro] -The sections below provide tutorials on the most frequently used and some less obvious features of {es}. - -For a full reference, see the {ref}/[Elasticsearch documentation] and in particular the {ref}/rest-apis.html[REST APIs] section. The {net-client} follows closely the JSON structures described there. - -A .NET API reference documentation for the Elasticsearch client package is available https://elastic.github.io/elasticsearch-net[here]. - -If you're new to {es}, make sure also to read {ref}/getting-started.html[Elasticsearch's quick start] that provides a good introduction. - -* <> -* <> -* <> - -NOTE: This is still a work in progress, more sections will be added in the near future. - -include::aggregations.asciidoc[] -include::esql.asciidoc[] -include::examples.asciidoc[] -include::mappings.asciidoc[] -include::query.asciidoc[] -include::recommendations.asciidoc[] -include::transport.asciidoc[] diff --git a/docs/usage/recommendations.asciidoc b/docs/usage/recommendations.asciidoc deleted file mode 100644 index b7f02a3589c..00000000000 --- a/docs/usage/recommendations.asciidoc +++ /dev/null @@ -1,37 +0,0 @@ -[[recommendations]] -== Usage recommendations - -To achieve the most efficient use of the {net-client}, we recommend following -the guidance defined in this article. - -[discrete] -=== Reuse the same client instance - -When working with the {net-client} we recommend that consumers reuse a single -instance of `ElasticsearchClient` for the entire lifetime of the application. -When reusing the same instance: - -- initialization overhead is limited to the first usage. -- resources such as TCP connections can be pooled and reused to improve -efficiency. -- serialization overhead is reduced, improving performance. - -The `ElasticsearchClient` type is thread-safe and can be shared and reused -across multiple threads in consuming applications. Client reuse can be achieved -by creating a singleton static instance or by registering the type with a -singleton lifetime when using dependency injection containers. - -[discrete] -=== Prefer asynchronous methods - -The {net-client} exposes synchronous and asynchronous methods on the -`ElasticsearchClient`. We recommend always preferring the asynchronous methods, -which have the `Async` suffix. Using the {net-client} requires sending HTTP -requests to {es} servers. Access to {es} is sometimes slow or delayed, and some -complex queries may take several seconds to return. If such operations are -blocked by calling the synchronous methods, the thread must wait until the HTTP -request is complete. In high-load scenarios, this can cause significant thread -usage, potentially affecting the throughput and performance of consuming -applications. By preferring the asynchronous methods, application threads can -continue with other work that doesn't depend on the web resource until the -potentially blocking task completes. \ No newline at end of file