Security?

[cxtal]

Hello and congratulations for an awesome project! The portal can be reached on the local network and, in case custom webpages are added that contain forms requiring credentials, then the credentials can be observed on the local network by just browsing to the device webserver.

Would there be some way to secure the entire portal? I know that ESP8266WebServer can support multiple authentication schemes such as BASIC, DIGEST, etc. but is there a way to implement that through portal.handleclient() elegantly?

Many thanks!

[Hieromon]

To secure the entire portal with the standard method of ESP8266WebServer, it is necessary to enhance the AutoConnect library.

BASIC authentication will be relatively easy to implement because AutoConnect knows its portal pages and the pages added on by AutoConnectAux. However, some users do not like this method because it is so simple. (It's better than nothing) So I have hesitated to implement.

When using MD5, we may get involved in the complex issues surrounding the captive portal. IETF95 has discussed the security issues of captive portals in the past.
https://tools.ietf.org/html/draft-nottingham-capport-problem-00
After all, the behavior of the client OS determines whether it can operate as intended as a captive portal. As of now, I haven't come up with any implementable way with ESP8266 that can solve this problem.

[cxtal]

Hi @Hieromon - thanks for replying! Please let me specify: the captive portal is not a problem. In the end, it has to be open so you can connect to it the first time and perform the initial configuration! That part is already secured by WPA2 and involves just the ESP and the connecting client without any third-parties (except those that would challenge WPA2. . .).

The problem is that the ESP can be accessed via the local network by browsing to its IP address in the local network. When the connection is opened to the IP address from the local network using a browser (ie: http://192.168.1.10/_ac), all the data is transferred without any sort of security.

If you have, say, passwords saved on the portal, they are transferred back and forth in plaintext, over the local network. This allows eavesdroppers to sniff the passwords easily. On a home network, maybe, that could perhaps not be a problem but if you are setting up a bunch of ESPs for a populated network, then most likely connecting to the IP address from the local network should be secured with at least DIGEST - I have seen that the ESP8266WebServer supports DIGEST: it is lightweight, involves an exchange of nonces and tokens between the browser and the server, should provide better security in the absence of SSL and should be supported by any browser.

Many thanks and sorry for the confusion!

[Hieromon]

@evacomaroski If you are talking about the ESP8266WebServer::authenticate and requestAuthenciation methods, Due to need to call the authenticate the requestAuthentication with each page, both BASIC and DIGEST of ESP8266WebServer are still an incomplete way for certification, there is no difference. If it should affect portal pages in the /_ac range including AutoConnectAux, I need to enhance the library as I mentioned earlier. Its enhancement will invoke authenticate and requestAuthenciation corresponding to the request handler of the library. (Both AutoConnect and PageBuilder)
However, this method has no effect on individual pages prepared by sketches. (Its substance is the request handler registered with the ESP8266WebServer::on function.) Conversely, if you want to pass individual credentials, it can be implemented outside of a custom web page with AutoConnectAux.

Here I want to confirm your wishes again.
What are the credentials you mentioned? for WiFi access-point which handled by AutoConnect? or for something else?
After all, do you want DIGEST authentication for custom web pages with AutoConnectAux?

[cxtal]

@Hieromon

Both BASIC and DIGEST authentications are cached in the browser [1] such that browsing to a different page on the same webserver will not require re-entering credentials. The difference between BASIC and DIGEST is that BASIC sends the password in plaintext whereas DIGEST encrypts the password [2] using some key-derivation scheme.

Succinctly, the problem is the following:

1. when the ESP is in AP mode, then a wireless client can connect to it and configure it
2. when the ESP is configured to connect to the local network, then a client on the local network can access the ESP webserver via the IP address and browse the pages.

In case (1), even though there is no authentication, connecting to the ESP when it is in AP mode (for the first time) and not connected to any network could be acceptable because it could be considered an initial configuration phase. Additionally, the credentials are transmitted between the ESP and the wireless client configuring the ESP such that sniffing the credentials is not possible. Sure, long-term ideas would be to generate a password that is based on some observable serial number for each ESP device [3].

In case (2), if any of the pages, both /_ac and custom pages created with AutoConnectAux contain any password forms, then any computer on the local network can see the passwords without even having to sniff the network. All that a computer on a local network would have to do, is scan the local network and check whether an "/_ac" URL is available, if so, the crawler could scan all the pages and discover pages containing sensitive information. It is true that after authentication, the information will be sent unencrypted unless SSL is added such that the credentials could be sniffed - however, not allowing access to the webserver in the first place is a step forward.

Here is an example of a custom webpage that I have created with AutoConnectAux following your documentation:

If you ever come to my house, all you have to do is find out the ESP IP, then you navigate with a browser to the AutoConnect portal and you can extract my MQTT password, username, IP address, port, etc. - none of which have a reason to be publicly accessible by anyone connected to the same network. Even so, you could do even more fun stuff by disrupting the ESP: reset it, make it connect to a different network, overwrite or vandalize the current settings, etc.

Now imagine that I am using AutoConnect for more serious stuff, let's say that I have many ESP devices spread out in a building with many employees and various network setups, each ESP containing custom pages with various sensitive information. You could get a lot of information by just accessing the AutoConnect portal and looking over the form fields.

As said earlier, this could be automated by writing a simple HTTP crawler:

1. scan the network IP block for available IPs, and for each IP,
2. check if IP/_ac is a valid URL (200),
3. crawl IP/_ac and, for all links with forms, extract all sensitive information

Again, without encryption (SSL), the information can be sniffed under certain circumstances, however finding a way to restrict access to the ESP webserver using some authentication would at least prevent access to the portal and any custom pages in the first place.

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Caching
[2] https://www.w3.org/Protocols/HTTP/digest_specification.html
[3] https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=201720180SB327

[Hieromon]

@evacomaroski, You may have misunderstood my question, but I have not denied all authentication support. I had already considered your point.
SSL is too heavy with ESP8266 and does not work properly with captive portal detection on some OSes. (I have tried it in the past and gave up, too little dynamic heap area left for user's sketch)
If it is a BASIC / DIGEST certification, AutoConnect can only influence the AutoConnectAux pages. It does not affect pages that are uniquely equipped with sketches. The parameters entered on the AutoConnectAux page can be passed to the ESP8266WebServer::on handler page which missing call ESP8266WebServer::authenticate. So, it is imperfectly, and I asked you a question.

After all, do you want DIGEST authentication for custom web pages with AutoConnectAux?

Is the value still useful to users?
I will always consider it positively if it is still worth it.
Do you have a way to be able to authenticate the page with the "on" function? If you have it, the idea is an important help for me.
I say agin, I have not denied the authentication support.
Thank you for your contribution.

[cxtal]

@Hieromon Yes, I imagined that SSL might be too heavy. Since I have started using AutoConnect, I went ahead and used only AutoConnectAux for custom webpages (ie: the screenshot from the previous comment).

After all, do you want DIGEST authentication for custom web pages with AutoConnectAux?

Yes. I have no other custom web pages that do not use AutoConnectAux. It keeps things consistent and unified in one single interface. The / (home) URL is not used at all for settings. So just /_ac would have to be protected since that is where the settings are made.

I do not mind much (in the absence of SSL) whether it is DIGEST or any other authentication: perhaps a simple page with Login and Password (or perhaps just password) should be sufficient? I mentioned DIGEST because I have seen it might be easy with the ESP8266 webserver and because it provides some additional protection over BASIC but I do not think it should matter too much.

Is the value still useful to users?

Imagine we work in the same place and I find some ESP devices with AutoConnect. I can just control them by entering their IP addresses plus /_ac and then I can turn off your ESP, I can modify the settings, I can reset it, perhaps you have OTA enabled too so I can reflash it, etc...

I guess that if users only use the ESP on their own personal network with only one user then there should be no problem. Otherwise, imagine you build a firm and use ESP devices to automate the doors or window blinds. Anyone on the local network could then just disable all your IoT devices.

Imagine a pub like StarBucks that would use AutoConnect: we can just go there, buy some coffee, connect to their "free wifi", scan for ESP devices and /_ac to determine if they are using it and then, not sure, can make their coffee machines go wild. It would be a tragedy without protection. :D

Do you have a way to be able to authenticate the page with the "on" function? If you have it, the idea is an important help for me.

I am pretty new to Arduino so I am not too sure that I can provide that kind of insight. However, as per the previous paragraph, even if you just add a "login form" - like, on a router login page, it is still much much better.

Thank you for considering it! The project is awesome! :-)

[Hieromon]

@evacomaroski, Thank you. I understood your intention cleary.
I start enhancement for DIGEST authentication for AutoConnect pages, including attached by AutoConnectAux as follows as a draft proposal for authentication support:

1. Add an option to AutoConnectConfig to enable authentication.
2. The authentication method is a DIGEST. BASIC authentication is also selectable to save runtime memory.
3. A user sketch can configure a scope of certification. i.e: Whether the certification range influences pages only attached by AutoConnectAux or including whole AutoConnect captive portal pages.
4. A user sketch cannot exclude for individual pages from a scope of certification. Just enable or disable authentication.

Please tell me your opinion.

[cxtal]

@Hieromon That looks very good! If it is too much of a hassle, even just a login form should at least prevent people from browsing / crawling for ESP devices on a network. :)

[Hieromon]

@evacomaroski This implementation is not too hard to work, but I am currently working for asynchronous AutoConnect. Combining it with ESPAsyncWebServer frees user sketches from the annoying headache of "When should call AutoConnect::handleClient" in a loop function.
However, even though I started implementation, I get the difficulty with thread synchronization with AsyncTCP on ESP32. It's tougher than expected.
So support for authentication may be a little delayed. Or maybe it's better to release the authentication feature first.
Let me think about it.

[cxtal]

@Hieromon Thank you for considering it! I am not sure about others but for me the security issue is critical. I live in a densely populated area where the local network is sort-of a mutual provider for an entire set of buildings. Connecting an ESP to such a network is pure suicide because right now anyone can access it and do anything with it. It might not be a problem for people living in remote and isolated areas but where I am the cables are piling on top of each other. :D I could wake up to all my devices going crazy... :D I get like 2-5 random bluetooth pairing requests on my phone alone if I leave bluetooth as discoverable....

Just saying so you can have an image in your head on the gravity of the situation...

[Hieromon]

@evacomaroski I staged the authentication support as v1.2.0 to the development branch.
You can evaluate it under the following conditions:

• AutoConenct 1.2.0 requires PageBuilder 1.4.0, which has not released yet. Please download manually and replace it with your libraries folder of Arduino/libraries/PageBuilder. Also, for AutoConnect 1.2.0 during staging with authentication support is the same.
  Or, Change a branch on your local repository after cloning to enhance/v120 for AutoConnect, to enhance/v140 for PageBuilder.

• I changed the configuration parameters a bit from the above draft. You can configure AutoConnectConfig as follows according to your requirements. It is achieved by a combination of two parameters, AutoConnectConfig::authentication and AutoConnectConfig::scope:

  ---

  1. Enable authentication
     AutoConnectConfig::authentication specifies enable HTTP authentication.
     • AC_AUTH_NONE : No authentication. (Default)
     • AC_AUTH_BASIC : Use HTTP authentication with BASIC method.
     • AC_AUTH_DIGEST : Use HTTP authentication with DIGEST method.
  2. Authentication scope
     AutoConnectConfig::scope specifies a scope of authentication influence pages.
     • AC_AUTHSCOPE_PORTAL : Whole AutoConnect portal pages including AutoConnectAux pages. (Not recommended)
     • AC_AUTHSCOPE_AUX : All AutoConnectAux pages. (Default)
     • AC_AUTHSCOPE_PARTIAL : Authenticate only specific AutoConnectAux pages which are specified by AutoConnectAux::authentication function or JSON.
  3. Authentication for the certain AutoConnectAux page with JSON
     You can specify the scope of authentication to certain pages as described below.
  4. Username and password
     You can specify a user name with AutoConnectConfig::username, password with AutoConnectConfig::password. Default values are same as APID/PSK. (Usually it is esp8266ap / 12345678 for esp8266)

• Not tested yet with ESP32.

Examples

1. Authentication for all AutoConnectAux pages.

AutoConnect portal;
AutoConnectConfig config;

void setup() {
  config.authentication = AC_AUTH_DIGEST;
  config.scope = AC_AUTHSCOPE_AUX; // Not necessary
  config.username = "admin";
  config.password = "password";
  portal.config(config);
  portal.begin();
}

2. Authentication with an only a certain page.

• Use AUtoConnectAux::authentication function.

Void AutoConnectAux::authentication(const AC_AUTH_t auth)

AC_AUTH_t type is enumeration with { AC_AUTH_NONE, AC_AUTH_DIGEST, AC_AUTH_DIGEST }.

• Specify with JSON.
  A key is auth, a value is none (default), basic or digest. (Ignore case)

{
  "title": "MQTT Setting",
  "uri": "/mqtt_setting",
  "menu": true,
  "auth": "digest",
  "element": [
  ]
}

• An example sketch

static const char AUX_mqtt_setting[] = R"(
[
  {
    "title": "MQTT Setting",
    "uri": "/mqtt_setting",
    "menu": true,
    "auth": "digest",
    "element": [
      {
        "name": "header",
        "type": "ACText",
        "value": "<h2>MQTT broker settings</h2>",
        "style": "text-align:center;color:#2f4f4f;padding:10px;"
      }
    ]
  },
  {
    "title": "MQTT Setting",
    "uri": "/mqtt_save",
    "menu": false,
    "element": [
      {
        "name": "caption",
        "type": "ACText",
        "value": "<h4>Parameters saved as:</h4>",
        "style": "text-align:center;color:#2f4f4f;padding:10px;"
      }
    ]
   }
]
)";

AutoConnect portal;
AutoConnectConfig config;

void setup() {
  config.authentication = AC_AUTH_DIGEST;
  config.scope = AC_AUTHSCOPE_PARTIAL;  // authentication for /mqtt_setting only
  config.username = "admin";
  config.password = "password";
  portal.config(config);
  portal.load(AUX_mqtt_setting);  // Loads pages that will be authenticated according to the "auth" key specified.
  portal.begin();
}

Limitations

• Authentication not works under the captive portal detection state. Due to ESP8266::requestAuthentication does not support 511 HTTP response.

However, testing is not enough.
If you have the time, please test it and report the results. It would be helpful for me.
Thank you for your contribution.

[cxtal]

@Hieromon that looks great! Will test for you with some WeMoS Mini devices - perhaps integrate it already in my current sketch. Many thanks!

[Hieromon]

@evacomaroski I found a problem during testing with AC_AUTH_PARTIAL also AC_AUTH_AUX such as should discuss with you.
If you access AutoConnectAux pages which are unauthorized yet while detecting the captive portal, then a 401 HTTP error will occur without prompting the login dialog on the web client. It is due to the condition of captive portals. When AutoConnect is under the captive portal state, it has not been authenticated to the network. That is, authentication via the upper protocol HTTP is not valid. This problem was expected as we already know, but I overlooked it for the AutoConnectAux pages behaving. If you want to enable HTTP authentication even if AutoConnect is under the captive portal state, I think unfortunately not.
So, I would like you to consider the followings about the AutoConnect to better behavior:

• AutoConnectAux pages that require authentication until the ESP8266 connects to any WiFi network are always unauthorized. i.e: You cannot enter secret parameters via AutoConnectAux pages unless you connect to any WiFi network.
  Can you accept this?

• The above-mentioned inconsistency of authentication behavior can be avoided by the user sketch.

  AutoConnect portal;
  AutoConnectConfig config;
  
  void setup() {
    WiFi.begin();
    if (WiFi.waitForConnectResult() != WL_CONNECTED)
      config.authentication = AC_AUTH_NONE;
    else
      config.authentication = AC_AUTH_PARTIAL;
    portal.config(config);
    portal.begin();
  }

  But this is a strange workaround.