missing feature: multithreading libs (like Due Scheduler)

[shiftleftplusone]

hello,
as these 2 Arduino multithreading libs
https://github.com/arduino-libraries/Scheduler (Zero, M0, Due)
https://github.com/mikaelpatel/Arduino-Scheduler (professed: AVR+ARM core)
do not work with ESP8266, which multithreading libs do you provide which work for the ESP8266 nodeMCU?

[lrmoreno007]

Ticker.h are directly inside this project, more info https://arduino-esp8266.readthedocs.io/en/latest/libraries.html#ticker

Ask these types of questions at www.esp8266.com

Please close this. It isn't a issue.

[shiftleftplusone]

no, I ask this question here because it's about a ported lib especially for the esp8266, such as the special libs about SD, SPI, Servo, Wire.
The missing library is the issue!
After all, @lrmoreno007 , I do not see either working multithreading lib in your links though, so what is the solution to my topic?

[shiftleftplusone]

so no multithreading available for the nodeMCU like the (cooperative) Scheduler or even a pre-emptive MT system, to be used as a lib with the standard Arduino IDE?

[devyte]

@shiftleftplusone the problem to implement this is dealing with the wifi stack context, which you don't need to do on Arduino or other devices. In the ESP, the wifi context is inside the espressif SDK.
I suspect it may be possible to do with some massive rewrite and reverse engineering of many low level routines, but that's just a guess, and the very few people who have the knowhow probably don't have the time or need.
Care to tackle it yourself?
Looking at the source code for the ESP's yield() and delay(), as well as the implementation of the libs you mention would be a starting point.
My engineering spidy sense hints at treating the wifi context as special, and treating all the loop contexts the same. When a loop context yield()s or delay()s, you store that loop's context and go back to the scheduler, who then immediately services the wifi. Then, once done, it chooses a loop of those available to continue. Rinse and repeat. Keep the contexts together with the loop info inside the scheduler.
You'd have to think about maintaining compatibility when the scheduler lib is not used, to avoid mem overhead in that case.

Async code executed from callbacks would still be forbidden from calling yield and delay.

The above is off the top of my head, and without deep knowledge of the code specifics, but I think it's doable.

[shiftleftplusone]

hi,
I am just an Arduino end user and I have actually absolutely no skills in C++ programming, WiFi contexts or whatever, or how MT for common Arduino is done (e.g., by some existing Schedulerlibs e.g. by cmaglie or mpatil for the Arduino Due). Nevertheless, multithreading is absolutely indispensible: WiFi can run in the background, and multiple threads (e.g. sensor watching by different polling speed via i2c, UART, SPI and controlling actors like motors and LEDs) must be able to be run simultaneously in parallel threads.

[devyte]

There are ways around what you seem to need without the Scheduler library. I am knowledgable in the field of automatic control, and I can tell you that you can implement control loops of differing frequencies, without multithreading. You just need polling on the cpu ticks and a roundrobin checker in the loop().
As an alternative, you may want to look into the FreeRTOS version of the ESP espressif SDK for real multithreading.

But let's not hijack this issue. You requested an implementation for the Scheduler. I would love to see it happen, but there doesn't seem to be anybody willing to pick up the request, so that leaves just you.
I could show you where to start looking. The code doesn't seem exactly easy to understand, but it's not big and broken into small pieces, which is always good. I also see a context container, currently of size 1, which would probably just need to be replaced, or extended and managed. There's also some docs from espressif.

If you're still not willing/able to pick up your own request, and learn interesting things about the ESP in the process, then please close the issue.

[shiftleftplusone]

I have been using the Scheduler libs by cmaglie for my Due and also the POSIX pthread libs for my Raspberry Pi (ANSI C, not C++) already very successfully since a lot of years and before that the MT libs of some different bytecode intepreters since almost 20 years for Lego Robots - I know how to handle multithreading, truely. But I never wrote MT libs from the scratch, neither for the Due nor POSIX libs for the Pi nor for Lego bytecode interpreters ever, and will honestly never do that for neither platform, and finally not for the nodeMCU: As stated, I am an end user, I expect those libs to be available for either platforms, ready-to-use for customers like me. So if no MT for the ESP8266 will be available, I will have to wait for them to come - or drop that node MCU.

[suculent]

The purpose of microcontrollers is to drive circuits on time. Multitasking and async processes do not have expectable timing and therefore do not fit such purpose and cannot be used in machine-control. You should select different core like RPi which has much more processing power. Moreover ESP8266 is a development platform and is not directly intended to be part of an end-user UX. Odesláno z iPhonu 4. 9. 2017 v 9:46, shiftleftplusone <notifications@github.com>:

…

I have been using the Scheduler libs by cmaglie for my Due and also the POSIX pthread libs for my Raspberry Pi (ANSI C, not C++) already very successfully since a lot of years and before that the MT libs of some different bytecode intepreters since almost 20 years for Lego Robots - I know how to handle multithreading, truely. But I never wrote Scheduler libs for the Due or POSIX libs for the Pi or bytecode interpreters for Lego bricks ever, and will honestly never do that for neither platform, and finally not for the nodeMCU: As stated, I am an end user, I expect those libs to be available for either platforms, ready-to-use for customers like me. So if no MT for the ESP8266 will be available, I will have to wait for them to come - or drop that node MCU. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[shiftleftplusone]

you seem not to be very experienced with the Arduino Due (ARM Cortex M3) - the (cooperative) MT Scheduler works fine and is indispensible for Robotics. Actually already the availability of different Scheduler libs (Patil, Maglie, Jensh, Pessaux, and more) both for the Due, M0, and partially even for AVR cores is proving it's eligibilily and legitimacy. The nodeMCU is almost as powerful as the Due , and much more powerful than AVRs and even better than M0 SAMD cores, and so MT libs are expected to be available, too, and thus to fit into the Arduino IDE concept plan for the Arduino target group.

[suculent]

You’re right. I’ll look at those first. I’m using mostly Extensa LX6 cores, most Arduinos are to slow for me and lack WiFi.

…

On 4 Sep 2017, at 10:01, shiftleftplusone ***@***.***> wrote: you seem not to be very experienced with the Arduino Due (ARM Cortex M3) - the (cooperative) MT Scheduler works fine and is indispensible for Robotics. Actually already the availability of different Scheduler libs (Patil, Maglie, and more) both for the Due, M0, and for AVR core is proving it's eligibilily and legitimacy. The nodeMCE is almost as powerful like the Due , and much more powerful than AVRs and even better than M0 SAMD cores, and so MT are expected to be available, too. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3390 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANix0wVZN77XUgpIo_d55Ip4yBgZkFUks5se65PgaJpZM4OLRyu>.

[suculent]

https://github.com/anmaped/esp8266-scheduler https://github.com/nrwiersma/ESP8266Scheduler Have you tried any of those? Odesláno z iPhonu 4. 9. 2017 v 10:01, shiftleftplusone <notifications@github.com>:

…

you seem not to be very experienced with the Arduino Due (ARM Cortex M3) - the (cooperative) MT Scheduler works fine and is indispensible for Robotics. Actually already the availability of different Scheduler libs (Patil, Maglie, and more) both for the Due, M0, and for AVR core is proving it's eligibilily and legitimacy. The nodeMCE is almost as powerful like the Due , and much more powerful than AVRs and even better than M0 SAMD cores, and so MT are expected to be available, too. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[devyte]

@suculent thank you for those links, I was not aware of those libs.

The first looks unmaintained for a couple of years, but looks rather advanced.
The second looks to be in active development. However, I find the fact that execution never reaches the main loop to be very scary.
I'm not sure how the interface compares to the standard Arduino Scheduler. Do we care about compatibility?

@shiftleftplusone care to try them both out and report back?

[suculent]

Examples and Arduino core system parts should be working on all models, which is kind of an argument against having this inside core OS library. However, @shiftleftplusone mentioned the major difference in ESP8266 and ‘standard’ Arduinos. In future, I’ll definitely use anything faster like ESP8266 (I have 20 of those), ESP32 or ILI7697. Arduino platform should help educating on multi-threading rather than being stuck to 8MHz Atmels (I have one in Arduino format and several nanos that are pretty useful, but WiFi-less).

…

On 5 Sep 2017, at 06:03, Develo ***@***.***> wrote: @suculent <https://github.com/suculent> thank you for those links, I was not aware of those libs. The first looks unmaintained for a couple of years, but looks rather advanced. The second looks to be in active development. However, I find the fact that execution never reaches the main loop to be very scary. I'm not sure how the interface compares to the standard Arduino Scheduler. Do we care about compatibility? @shiftleftplusone <https://github.com/shiftleftplusone> care to try them both out and report back? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3390 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANix0whMiw6uUZ0i1nZzaKiVn3j3DKBks5sfMgEgaJpZM4OLRyu>.

[shiftleftplusone]

hello,
the anmaped/esp8266-scheduler seems to be reasonable as it's forked from mikaelpatel/Arduino-Scheduler which I once already tested on my Due. Although some issues have already been reported for the original fork run by Dues (opposite to the cmaglie scheduler which always works fine) it seems to be the more promising approach for the esp8266 IMO.

The 2nd one (https://github.com/nrwiersma/ESP8266Scheduler) using classes is too weird tbh.

Most promising for the future would be an approach based on POSIX pthread ported to Arduino though.

I'll have a look at the mpatil(anmaped scheduler and report ASAP.