ENV and IDBFS are undefined in 1.38.47-upstream & above

[lpotter]

Reporting an issue we are seeing as in the Qt bug report:
https://bugreports.qt.io/browse/QTBUG-79971

ENV and IDBFS javascript use in upstream (1.38.47, 1.39.2) do not work as they did, and result in the message:

ReferenceError: ENV is not defined qtloader.js:411:17

[kripken]

The IDBFS issue is a change in what is exported by default, see the changelog,

https://github.com/emscripten-core/emscripten/blob/incoming/ChangeLog.md#v1391-10302019

(So you need to add -lidbfs.js during link.) Sorry for the inconvenience!

There was a recent ENV refactor, but there should not be any user-visible changes (in particular, the presence of getenv or setenv in the compiled program should automatically link in the JS ENV object). How do you use the ENV object, and what environment C functions are used?

[lpotter]

We use ::getenv on the Qt side.

and ENV like such:

Module.preRun.push(function() {
for (var [key, value] of Object.entries(config.environment)) {
ENV[key.toUpperCase()] = value;
}
});

[kripken]

Hmm, that should work. And we have similar things in the test suite: https://github.com/emscripten-core/emscripten/blob/incoming/tests/test_other.py#L6461

Perhaps you can make a testcase I can take a look at?

[kleisauke]

I have the same issue with setting environment variables in combination with -s MODULARIZE=1. Here is a test case that reproduces this problem:

  def test_modularize_override_c_environ(self):
    src = r'''
      #include <stdlib.h>
      #include <string.h>
      #include <assert.h>
      int main() {
        const char *hello = getenv("hello");
        assert(strcmp(hello, "world") == 0);
      #ifdef REPORT_RESULT
        REPORT_RESULT(0);
      #endif
        return 0;
      }
    '''
    create_test_file('test.c', self.with_report_result(src))
    self.compile_btest(['test.c', '-s', 'MODULARIZE=1'])

    create_test_file('a.html', '''
      <script src="a.out.js"></script>
      <script>
        var options = {
          noInitialRun: true,
          preRun: [function() { ENV.hello = 'world' }]
        };
        Module(options).then(function(hello) {
          hello._main();
        });
      </script>
    ''')
    self.run_browser('a.html', None, '/report_result?0')

Results in:

ReferenceError: ENV is not defined

As a workaround I'm currently doing this:

Workaround

  @no_wasm_backend('does not seem to work in the Wasm backend')
  def test_modularize_override_c_environ_workaround(self):
    src = r'''
      #include <stdlib.h>
      #include <string.h>
      #include <assert.h>
      int main() {
        const char *hello = getenv("hello");
        assert(strcmp(hello, "world") == 0);
      #ifdef REPORT_RESULT
        REPORT_RESULT(0);
      #endif
        return 0;
      }
    '''
    create_test_file('test.c', self.with_report_result(src))
    self.compile_btest(['test.c', '-s', 'MODULARIZE=1', '-s', 'EXTRA_EXPORTED_RUNTIME_METHODS=["ENV"]'])

    create_test_file('a.html', '''
      <script src="a.out.js"></script>
      <script>
        var options = {
          noInitialRun: true,
        };
        Module(options).then(function(hello) {
          hello.ENV.hello = 'world';
          hello._main();
        });
      </script>
    ''')
    self.run_browser('a.html', None, '/report_result?0')

But this only seems to work for the fastcomp backend.

[kripken]

@kleisauke I think this is because of how MODULARIZE works. Looking in the html and js output here, your html has that reference to ENV, which will only work if ENV is in the global scope. But in MODULARIZE mode it isn't in the global scope any more.

The only thing that can refer to ENV is something in the function scope of MODULARIZE. So you might use a pre-js to add a preRun to change ENV (if you want it to get the values from the html, then you'd need to get that data into where it can see it, perhaps by passing it on Module for example).

[kripken]

btw @kleisauke thank you for the nice testcase - it made it very quick to compile and investigate! 😄

[kleisauke]

Thanks for the explanation! I'm glad that the testcase was useful. 😄

For further readers; you can use this to set environment variables in combination with -s MODULARIZE=1:

postamble_env.js:

addOnPreRun(function () {
    if (Module['env'] && typeof Module['env'] === 'object') {
        for (var key in Module['env']) {
            if (Module['env'].hasOwnProperty(key))
                ENV[key] = Module['env'][key];
        }
    }
});

Compile with:

emcc test.c -s MODULARIZE=1 --post-js postamble_env.js -o test.html

Usage:

var options = {
    env: {
        hello: 'world'
    }
};
Module(options).then(function (module) {

});

[Ecco]

Great! Question: how about merging @kleisauke 's suggestion? This seems like a great fix that everyone could use!

[kleisauke]

An alternative way is to pass the module as the first argument when building with MODULARIZE=1. See for example this patch:

Patch

From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Kleis Auke Wolthuizen <github@kleisauke.nl>
Date: Sun, 9 Feb 2020 16:00:00 +0100
Subject: [PATCH 1/1] Pass Module to callback functions when MODULARIZE=1

This ensures that users can still invoke runtime elements within the
preRun function that are exported on Module (for e.g. "ENV" or "FS").

See: emscripten-core/emscripten#9827

diff --git a/src/preamble.js b/src/preamble.js
index 1111111..2222222 100644
--- a/src/preamble.js
+++ b/src/preamble.js
@@ -452,7 +452,12 @@ function callRuntimeCallbacks(callbacks) {
   while(callbacks.length > 0) {
     var callback = callbacks.shift();
     if (typeof callback == 'function') {
+// In modularize mode, we pass the module as the first argument
+#if MODULARIZE
+      callback(Module);
+#else // !MODULARIZE
       callback();
+#endif // !MODULARIZE
       continue;
     }
     var func = callback.func;

Usage:

var options = {
    preRun: [function(module) {
        module.ENV.hello = 'world';
        module.FS.createPreloadedFile('/', 'foo', 'bar', true, false);
    }]
};
Module(options).then(function (module) {

});

[Ecco]

Yes indeed, this would work too! Perhaps even better since it allows manipulating all globals (ENV, FS, maybe others?)

[kripken]

Nice idea and good points @kleisauke and @Ecco !

@kleisauke maybe you can open a PR with that last suggestion, so it gets a larger discussion? Note that one issue we'll need to consider carefully is that in #10697 the API here may be changing, and I'm not sure offhand if that interacts with this or not.