XIAO ESP32-C3: I2S takes 250ms before a INMP441 microphone can give accurate readings with I2S.

[zackees]

Confirmed working in esp32 idf 5.1 alpha 3.0.0

The INMP441 mems mic is the culprit. The working theory is that when the I2S clock stops the microphone goes into a power down mode and then generates noise during power up when I2S resumes. See final comment for possible work around using LEDC to generate a pseudo clock to keep the device awake.

Example:

Now entering light sleep
Exited light sleep, now outputting sound levels for one second.
vol: 2874
vol: 1315
vol: 579
vol: 887
vol: 764
vol: 587
vol: 399
vol: 270
vol: 231
vol: 134
vol: 50
vol: 68
vol: 50
vol: 31
vol: 38
vol: 36
vol: 41
vol: 35
vol: 33
vol: 47

Board

XIAO ESP32-C3 with battery

Device Description

https://www.amazon.com/dp/B0B94JZ2YF?psc=1&ref=ppx_yo2ov_dt_b_product_details

I'm using Platform io. Here is my ini file

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter
;   Upload options: custom upload port, speed and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html

[env:esp32c3]
platform = espressif32@^6.2.0
board = seeed_xiao_esp32c3
framework = arduino

; change microcontroller
board_build.mcu = esp32c3
;upload_speed = 115200

; change WiFi firmware
board_build.variant = esp32c3

monitor_speed = 115200

board_build.f_cpu = 80000000L

build_flags = 
    -D CONFIG_PM_ENABLE
    -D CONFIG_PM_USE_RTC
    -D CONFIG_PM_LIGHTSLEEP_RTC_OSC_CAL_INTERVAL=8

I have a INMP441 mems mic:

https://www.amazon.com/AITRIP-Omnidirectional-Microphone-Precision-Interface/dp/B092HWW4RS/ref=sr_1_3?keywords=INMP441&sr=8-3

For the life of me I cannot get the IS2 peripheral to power back on quickly. I have to wait about 1/3rd of a second before the microphone stabilizes. I've tried so many different things: changing the clock signal. Changing the board frequency. Nothing seems to work to make the microphone better.

Hardware Configuration

#ifndef _DEFS_H_
#define _DEFS_H_

#include <iostream>
using std::cout;
using std::endl;

#define MAX_ANALOG_READ 1023

#define LED_PIN 9  // Pin is held high on startup.
#define LED_PIN_IS_SINK true
#define LED_0 0  // Led channel name
#define LED_PWM_FREQ 2000
#define LED_PWM_RESOLUTION 14  // Max on ESP32-c3 XIOA

#define TIME_PWM_CYCLE_MS  3 // Flickers at 1ms
#define TIME_PWM_TRANSITION_MS 3  // 60 fps

#define PIN_I2S_WS GPIO_NUM_3  // TODO change this pins
#define PIN_IS2_SD GPIO_NUM_2  // TODO change this pins
#define PIN_I2S_SCK GPIO_NUM_4  // TODO change this pins
#define PIN_AUDIO_PWR GPIO_NUM_5  // TODO change this pins

#define I2S_NUM I2S_NUM_0
// #define IS2_AUDIO_BUFFER_LEN 1024 // max samples for i2s_read
#define IS2_AUDIO_BUFFER_LEN 1024 // max samples for i2s_read
#define AUDIO_BIT_RESOLUTION 16
#define AUDIO_SAMPLE_RATE (44100ul / 1)
#define AUDIO_CHANNELS 1 // Not tested with 2 channels
#define AUDIO_DMA_BUFFER_COUNT 3
#define AUDIO_RECORDING_SECONDS 1

#define TIME_BEFORE_LIGHT_SLEEP_MS 1000
#define LIGHT_SLEEP_TIME_uS uint32_t(1000 * 100) // 100 ms.


#define ASSERT_IMPL(x, msg, file, lineno)                                                                       \
  do                                                                                                            \
  {                                                                                                             \
    if (!(x))                                                                                                   \
    {                                                                                                           \
      std::cout << "#############\n# ASSERTION FAILED: " << file << ":" << lineno << "\n# MSG: " << msg << "\n#############\n"; \
      configASSERT(x);                                                                                          \
    }                                                                                                           \
  } while (false);

#define ASSERT(x, msg) ASSERT_IMPL(x, msg, __FILE__, __LINE__)


#endif  // _DEFS_H_

Version

v2.0.9

IDE Name

PlatformIO

Operating System

Max OS 13 on M1

Flash frequency

default

PSRAM enabled

no

Upload speed

115200

Description

I2S has to wait a long time before it stabilizes after a light sleep. I have to throw away the next 12 buffers of 1024, 44100 hz audio data in mono format. I thought it was the microphone but it appears to be the IS2 bus.

Sketch

++

#include <iostream>

#include "audio.h"
#include "defs.h"
#include <Arduino.h>
#include <stdint.h>
#include <driver/i2s.h>
#include "ringbuffer.hpp"
#include "alloc.h"
#include "buffer.hpp"
#include "task.h"
#include <limits>
#include "time.h"
#include <stdio.h>
#include <atomic>

#include "alloc.h"

using namespace std;

#define ENABLE_AUDIO_TASK 0

#define AUDIO_TASK_SAMPLING_PRIORITY 7

#define AUDIO_BUFFER_SAMPLES (AUDIO_RECORDING_SECONDS * AUDIO_SAMPLE_RATE * AUDIO_CHANNELS)

// During power
#define POWER_ON_TIME_MS 85  // Time to power on the microphone according to the datasheet.
#define POWER_OFF_TIME_MS 85  // Time to power off the microphone is 43 ms but we round up.
                              // Note that during power down, no data should be attempted to be read
                              // or the ESD diodes will be activated and the microphone will be damaged.

namespace
{
  static_assert(AUDIO_BIT_RESOLUTION == 16, "Only 16 bit resolution is supported");
  static_assert(AUDIO_CHANNELS == 1, "Only 1 channel is supported");
  static_assert(sizeof(audio_sample_t) == 2, "audio_sample_t must be 16 bit");
  std::atomic<float> s_loudness_dB;
  std::atomic<uint32_t> s_loudness_updated;
  int garbage_buffer_count = 0;

  const i2s_config_t i2s_config = {
      .mode = i2s_mode_t(I2S_MODE_MASTER | I2S_MODE_RX),
      .sample_rate = AUDIO_SAMPLE_RATE,
      .bits_per_sample = i2s_bits_per_sample_t(AUDIO_BIT_RESOLUTION),
      .channel_format = i2s_channel_fmt_t(I2S_CHANNEL_FMT_ONLY_RIGHT),
      .communication_format = i2s_comm_format_t(I2S_COMM_FORMAT_STAND_I2S),
      .intr_alloc_flags = 0,
      .dma_buf_count = AUDIO_DMA_BUFFER_COUNT,
      .dma_buf_len = IS2_AUDIO_BUFFER_LEN,
      .use_apll = false,
      //.tx_desc_auto_clear = true,
      //.fixed_mclk = 4000000ul,
  };

  const i2s_pin_config_t pin_config = {
      .bck_io_num = PIN_I2S_SCK,
      .ws_io_num = PIN_I2S_WS,
      .data_out_num = I2S_PIN_NO_CHANGE,
      .data_in_num = PIN_IS2_SD};

  void i2s_audio_init()
  {

    i2s_driver_install(I2S_NUM_0, &i2s_config, 0, NULL);
    i2s_set_pin(I2S_NUM, &pin_config);
    //i2s_zero_dma_buffer(I2S_NUM_0);
    //i2s_start(I2S_NUM_0);
  }

  void i2s_audio_shutdown()
  {
    //i2s_stop(I2S_NUM_0);
    i2s_driver_uninstall(I2S_NUM_0);
  }



  double audio_loudness_to_dB(double rms_loudness)
  {
    // This is a rough approximation of the loudness to dB scale.
    // The data was taken from the following video featuring brown
    // noise: https://www.youtube.com/watch?v=hXetO_bYcMo
    // This linear regression was done on the following data:
    // DB | LOUDNESS
    // ---+---------
    // 50 | 15
    // 55 | 22
    // 60 | 33
    // 65 | 56
    // 70 | 104
    // 75 | 190
    // 80 | 333
    // This will produce an exponential regression of the form:
    //  0.0833 * std::exp(0.119 * x);
    // Below is the inverse exponential regression.
    const float kCoefficient = 0.119f;
    const float kIntercept = 0.0833f;
    const float kInverseCoefficient = 1.0f / kCoefficient; // Maybe faster to precompute this.
    const float kInverseIntercept = 1.0f / kIntercept;
    return std::log(rms_loudness * kInverseIntercept) * kInverseCoefficient;
  }

  float calc_rms_loudness(const audio_sample_t *samples, size_t num_samples)
  {
    uint64_t sum_of_squares = 0;
    for (size_t i = 0; i < num_samples; ++i)
    {
      sum_of_squares += samples[i] * samples[i];
    }
    double mean_square = static_cast<double>(sum_of_squares) / num_samples;
    return static_cast<float>(std::sqrt(mean_square));
  }

  size_t read_raw_samples(audio_sample_t (&buffer)[IS2_AUDIO_BUFFER_LEN])
  {
    size_t bytes_read = 0;
    i2s_event_t event;

    uint32_t current_time = millis();
    esp_err_t result = i2s_read(I2S_NUM_0, buffer, sizeof(buffer), &bytes_read, 0);
    if (result == ESP_OK)
    {
      if (bytes_read > 0)
      {
        //cout << "Bytes read: " << bytes_read << endl;
        const size_t count = bytes_read / sizeof(audio_sample_t);
        return count;
      }
    }
    return 0;
  }

  bool update_audio_samples()
  {
    audio_sample_t buffer[IS2_AUDIO_BUFFER_LEN] = {0};
    bool updated = false;
    while (true) {
      size_t samples_read = read_raw_samples(buffer);
      if (samples_read <= 0)
      {
        break;
      }
      if (garbage_buffer_count > 0) {
        --garbage_buffer_count;
        continue;
      }
      updated = true;
      float rms = calc_rms_loudness(buffer, samples_read);
      s_loudness_dB.store(audio_loudness_to_dB(rms));
      s_loudness_updated.store(millis());
    }
    return updated;
  }
  bool s_audio_initialized = false;
} // anonymous namespace

void audio_task(void *pvParameters)
{
  while (true)
  {
    // Drain out all pending buffers.
    while (update_audio_samples())
    {
      ;
    }
    delay_task_ms(7);
  }
}

void audio_init(bool wait_for_power_on)
{
  if (s_audio_initialized)
  {
    cout << "Audio already initialized." << endl;
    return;
  }
  s_audio_initialized = true;
  pinMode(PIN_AUDIO_PWR, OUTPUT);
  digitalWrite(PIN_AUDIO_PWR, HIGH);  // Power on the IS2 microphone.
  i2s_audio_init();
  if (wait_for_power_on) {
    delay_task_ms(POWER_ON_TIME_MS);  // Wait for the microphone to power on.
  }


  // start a task to read the audio samples using psram
  //TaskCreatePsramPinnedToCore(
  //    audio_task, "audio_task", 4096, NULL, AUDIO_TASK_SAMPLING_PRIORITY, NULL, 0);

  #if ENABLE_AUDIO_TASK
  xTaskCreatePinnedToCore(
      audio_task, "audio_task", 4096, NULL, AUDIO_TASK_SAMPLING_PRIORITY, NULL, 0);
  #endif
}

  // UNTESTED
  void audio_shutdown()
  {
    //i2s_stop(I2S_NUM_0);          // Stop the I2S
    //i2s_driver_uninstall(I2S_NUM_0); // Uninstall the driver
    s_audio_initialized = false;
  }


audio_state_t audio_update()
{
  uint32_t start_time = millis();
  update_audio_samples();
  #if ENABLE_AUDIO_TASK
  for (int i = 0; i < 3; i++)
  {
    vPortYield();
  }
  #endif
  audio_state_t state = audio_state_t(audio_loudness_dB(), s_loudness_updated.load());
  return state;
}

float audio_loudness_dB() { return s_loudness_dB.load(); }


// Audio

void audio_loudness_test()
{
  Buffer<double> sample_buffer;
  sample_buffer.init(32);
  cout << "Done initializing audio buffers" << endl;

  while (true)
  {
    // This is a test to see how loud the audio is.
    // It's not used in the final product.
    audio_sample_t buffer[IS2_AUDIO_BUFFER_LEN] = {0};
    size_t samples_read = read_raw_samples(buffer);
    if (samples_read > 0)
    {
      double rms_loudness = calc_rms_loudness(buffer, samples_read);
      sample_buffer.write(&rms_loudness, 1);
      double avg = 0;
      for (size_t i = 0; i < sample_buffer.size(); ++i)
      {
        avg += sample_buffer[i];
      }
      avg /= sample_buffer.size();
      String loudness_str = String(avg, 3);
      // Serial.printf("Avg rms loudness: %s\n", loudness_str.c_str());
      float dB = audio_loudness_to_dB(avg);
      String dB_str = String(dB, 3);
      Serial.printf("dB: %s, loudness: %s\n", dB_str.c_str(), loudness_str.c_str());
      // buffer->clear();
      sample_buffer.clear();
    }
  }
}

void audio_enter_light_sleep() {
  audio_sample_t buffer[IS2_AUDIO_BUFFER_LEN] = {0};
  //i2s_stop(I2S_NUM_0);          // Stop the I2S
  i2s_audio_shutdown();
  pinMode(PIN_I2S_SCK, OUTPUT);  // This is all desperation trying to reset pin state so that when I2S restarts it works.
  digitalWrite(PIN_I2S_SCK, LOW);
  pinMode(PIN_I2S_WS, OUTPUT);
  digitalWrite(PIN_I2S_WS, LOW);
  pinMode(PIN_IS2_SD, OUTPUT);
  digitalWrite(PIN_IS2_SD, LOW);
  // i2s_zero_dma_buffer(I2S_NUM_0);
  //digitalWrite(PIN_AUDIO_PWR, HIGH);
  gpio_hold_en(PIN_I2S_WS);
  gpio_hold_en(PIN_IS2_SD);
  gpio_hold_en(PIN_I2S_SCK);
  gpio_hold_en(PIN_AUDIO_PWR);
}

void audio_exit_light_sleep() {

  //delay(8);
  i2s_audio_init();
  gpio_hold_dis(PIN_I2S_WS);
  gpio_hold_dis(PIN_IS2_SD);
  gpio_hold_dis(PIN_I2S_SCK);
  // gpio_hold_dis(PIN_AUDIO_PWR);

  // i2s_start(I2S_NUM_0);
  i2s_audio_init();
  //delay(160);
  //audio_sample_t buffer[IS2_AUDIO_BUFFER_LEN] = {0};
  //uint32_t future_time = millis() + 180;
  //while (millis() < future_time) {
  //  read_raw_samples(buffer);
  //}
  //delay(180);
  //delay(POWER_ON_TIME_MS * 2);
  garbage_buffer_count = 14; // For some reason it takes a few buffers to get the audio going again.
}

Debug Message

None. But the recorded sound levels are through the roof for the first few buffers then decrease.

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[me-no-dev]

Since you are using ESP-IDF's API, I suggest you file an issue in their repo. You can say that you are running on ESP-IDF v4.4.4

[SuGlider]

My wild guess about it is that entering/returning from light sleep changes the source frequency of the I2S peripheral.
Thus, the first 300ms, the I2S peripheral may be running in the wrong frequency and it looks like reading garbage data up to the system starts back to run in the correct APB Freq.

Maybe (just a guess) if the I2S is set to use a Clock Source that is not APB dependent, it may not be affected by this change.

.use_apll = true, instead of false?

The new IDF 5.1 I2S driver documentation talk about using APLL... (but remember that Arduino 2.0.9 uses IDF 4.4.4).
This is just for reading and understanding my "wild guess".
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/peripherals/i2s.html#i2s-clock

[SuGlider]

Anyway.... this exactly same issue also happens with UART when returning from Light Sleep. The first bytes read are bad...
It is documented. Maybe.... it just needs to document that I2S has a similar behavior, like the one you are experimenting.

[zackees]

I used .use_apll = true as well and used a fixed clock frequency. I also tried different CPU frequencies. It seems that 80 MHz is better and I'm able to minimize the number of 1k buffers I have to discard (1024@44.1khz, single channel, 16-bit) down from 16 to about 10. Adjusting the frame down to 512, 128 bytes doesn't seem to affect the length of time that the I2S is generating bad data.

It's a shame because the MEMS mic performs so well otherwise. I tried putting a medium-sized ceramic decoupling capacitor at the power pins of the MEMS, but this didn't do anything.

If I want to get proper light sleep I'll have to use an analog microphone and sample it with an ISR.

[SuGlider]

I guess that those 300ms after returning from light sleep are very important for the project.
Why using an analog microphone would make it work?

Can you imagine a process/work around that fits better the needs of the project?
Stopping the I2S before going into light sleep and starting it after returning would work?

[zackees]

It works because I can just keep the analog microphone on in a powered state while the CPU sleeps and then immediately start reading again after the processor wakes.

This will allow me to microsleep the CPU in the main loop and immediately execute the next loop iteration without a delay. This should massively reduce power consumption.

[SuGlider]

I'm not sure that ESP32 ADC will not also take some time to read accurate data...
It uses SAR algorithm and it may also depend on the clock source.

A potential reference: espressif/esp-idf#8287

[SuGlider]

A possible alternative strategy to go low power and get MIC data could be to use the ULP to read the data... but the C3 has no ULP.
It could be possible with an S3 instead, using its RISC-V ULP.
https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-reference/system/ulp-risc-v.html

[SuGlider]

Another way to think about low power, could be run the C3 at 10MHz, no sleep needed.
It may be possible to read the I2S MIC at this CPU clock and maybe it consumes very low current.
At 10MHz, Wifi and BLE won't work... but if necessary, it can change the CPU frequency back to 80MHz in order to send some data to a server, and then back to 10MHz to read the MIC... just thinking here.

All of it will need experimentation and measures.

[zackees]

It's not possible to read the I2S under 80 mhz.

[SuGlider]

It's not possible to read the I2S under 80 mhz.

With some work to code it... RMT may do it. Not easy... but it may be possible to decode an I2S MIC signal...
Anyway, it may be a huge challenge and not work properly after a big effort to code it. Too risky, maybe.

[zackees]

Sounds complicated. Reading from an analog condenser mic is well known and the components are cheap. I only need it for loudness and some fft. While the mems produced excellent audio, the condenser mic will work well enough.