[8.0] Allow using http_auth with RequestsHttpNode for custom authentication

[sethmlarson]

Using http_auth is deprecated for passing Tuple[str, str] and str but is useful for custom HTTP authentication with the Requests library. For example Kerberos and GSS are possible with this method. To allow for these to continue working we'll continue setting requests.Session.auth = http_auth when used this way.

Reference: #1776 (comment)

[sethmlarson]

Work is done on the transport side: elastic/elastic-transport-python#48

[naisanzaa]

Error importing RequestsHttpConnection:

from elasticsearch import RequestsHttpConnection

Traceback (most recent call last):
  File "/Users/eric/venv/garbage/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3251, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-8-6f544e924cdd>", line 1, in <module>
    from elasticsearch import RequestsHttpConnection
ImportError: cannot import name 'RequestsHttpConnection' from 'elasticsearch' (/Users/eric/venv/garbage/lib/python3.9/site-packages/elasticsearch/__init__.py)

elastic-transport==8.0.1
elasticsearch==8.0.0

[sethmlarson]

@naisanzaa Transport-layer classes aren't supplied via the elasticsearch package any more, the elastic_transport package is used for multiple packages including elasticsearch. You can find RequestsHttpNode in that package.

[Nour963]

So what is the solution to this problem ?
I'm still having the same issue, with ES 8.1.0
This stackoverflow post has an answer that says to downgrade from python package elasticsearch==8.0.0 to elasticsearch==7.13.4

[sethmlarson]

Since there's a lot of interest in this feature, this will be released in 8.2.0 but support is currently merged into main.

[james-henry-git]

So what is the solution to this problem ? I'm still having the same issue, with ES 8.1.0 This stackoverflow post has an answer that says to downgrade from python package elasticsearch==8.0.0 to elasticsearch==7.13.4

FYI, for those struggling with using the requests module in Elasticsearch 8.x is hardly documented well, we shouldn't need to thunk down versions, we just need to tell it to use RequestsHttpNode and set the appropriate variables.

import os
os.environ["https_proxy"] = "http://proxy_name:8080"
from elastic_transport import RequestsHttpNode
from elasticsearch import Elasticsearch
client = Elasticsearch(
    cloud_id = authkeys['cloud_id'], 
    api_key = authkeys['api_key'],
    node_class=RequestsHttpNode,
    )

[Adam-Sparrow]

So I am currently studying a course and this weeks focus is elasticsearch. Unfortunately the course materials provide us the import code which is failing in this thread.

When I use the os.environ["https_proxy"] line, do I need to supply a proxy_name?

I also don't have cloud_id and an api_key to my knowledge unless that is my user and password secrets.

Here is the code provided by my course materials, I will direct them to this thread to highlight the need for an update.

They had us install version 7.14, but the issue persisted in my instance. That is likely my fault, but still it seems I am being taught something out of date for at least 2.5 years.

from elasticsearch import Elasticsearch
from elasticsearch import RequestsHttpConnection
import hidden

# Load the secrets
secrets = hidden.elastic()

es = Elasticsearch(
    [ secrets['host'] ],
    http_auth=(secrets['user'], secrets['pass']),
    url_prefix = secrets['prefix'],
    scheme=secrets['scheme'],
    port=secrets['port'],
    connection_class=RequestsHttpConnection,
)