undefined method `exception' for nil:NilClass with Ruby 3.1.0

[aaronjensen]

* Operating system:                 mac 
* Ruby implementation:             Ruby 3.1.0
* `concurrent-ruby` version:       1.1.9
* `concurrent-ruby-ext` installed:  no
* `concurrent-ruby-edge` used:     no

After upgrading to Ruby 3.1.0 I'm getting a very strange, very hard (for me) to debug error with premailer+sprockets that ultimately manifests with concurrent failing to provide a reason for a promise rejection.

The error is strange enough that it seems like it could be a ruby bug.

When stepping through with byebug I get here:

https://github.com/rails/sprockets/blob/v4.0.2/lib/sprockets/manifest.rb#L143

Then, drilling into asset.source, it returns the source fine. The yield however causes a jump all the way out of the begin in SafeTaskExecutor#execute. If I put an ensure on that begin, it runs. Nothing after @task.call in the block runs. No exception can be caught.

I then get this:

        project/gems/ruby/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/concern/obligation.rb:128:in `exception': undefined method `exception' for nil:NilClass

        reason.exception(*args)
              ^^^^^^^^^^ (NoMethodError)
                from project/gems/ruby/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/concern/obligation.rb:87:in `raise'
                from project/gems/ruby/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/concern/obligation.rb:87:in `block in wait!'
                from <internal:kernel>:90:in `tap'
                from project/gems/ruby/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/concern/obligation.rb:87:in `wait!'
                from project/gems/ruby/3.1.0/gems/sprockets-4.0.2/lib/sprockets/manifest.rb:141:in `each'
                from project/gems/ruby/3.1.0/gems/sprockets-4.0.2/lib/sprockets/manifest.rb:141:in `find'
                from project/gems/ruby/3.1.0/gems/sprockets-4.0.2/lib/sprockets/manifest.rb:153:in `each'
                from project/gems/ruby/3.1.0/gems/sprockets-4.0.2/lib/sprockets/manifest.rb:153:in `find_sources'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_loaders/asset_pipeline_loader.rb:13:in `each'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_loaders/asset_pipeline_loader.rb:13:in `first'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_loaders/asset_pipeline_loader.rb:13:in `load'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:48:in `block in load_css'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:46:in `each'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:46:in `load_css'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:20:in `css_for_url'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:13:in `block in css_for_doc'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:13:in `map'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:13:in `css_for_doc'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/customized_premailer.rb:14:in `initialize'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:93:in `new'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:93:in `premailer'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:74:in `generate_html_part'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:52:in `generate_html_part_replacement'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:24:in `perform'
                from project/gems/ruby/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:8:in `perform'
 

I may be able to narrow down a repro if that would be useful, but I wanted to file this now in case there are already any known issues or work arounds I'm not aware of.

Thanks!

[agrberg]

Problem

I'm having the same error via premailer-rails as well and have been looking into it a bit myself this morning. My difference is that block_given? in Sprockets::Manifest#find_sources is false so I have an instance of Enumerable in https://github.com/fphilipe/premailer-rails/blob/v1.11.1/lib/premailer/rails/css_loaders/asset_pipeline_loader.rb#L11. Calling first on that object still causes the same error seen here even though the object responds to the method.

/.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/concern/obligation.rb:128:in `exception': undefined method `exception' for nil:NilClass (NoMethodError)

        reason.exception(*args)
              ^^^^^^^^^^
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/concern/obligation.rb:87:in `raise'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/concern/obligation.rb:87:in `block in wait!'
	from <internal:kernel>:90:in `tap'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/concurrent-ruby-1.1.9/lib/concurrent-ruby/concurrent/concern/obligation.rb:87:in `wait!'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/sprockets-4.0.2/lib/sprockets/manifest.rb:130:in `each'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/sprockets-4.0.2/lib/sprockets/manifest.rb:130:in `find'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/sprockets-4.0.2/lib/sprockets/manifest.rb:142:in `each'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/sprockets-4.0.2/lib/sprockets/manifest.rb:142:in `find_sources'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_loaders/asset_pipeline_loader.rb:12:in `each'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_loaders/asset_pipeline_loader.rb:12:in `first'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_loaders/asset_pipeline_loader.rb:12:in `load'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:47:in `block in load_css'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:46:in `each'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:46:in `load_css'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:20:in `css_for_url'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:13:in `block in css_for_doc'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:13:in `map'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:13:in `css_for_doc'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/customized_premailer.rb:14:in `initialize'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:93:in `new'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:93:in `premailer'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:84:in `generate_text_part'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:62:in `generate_alternative_part'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:50:in `generate_html_part_replacement'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:24:in `perform'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/hook.rb:8:in `perform'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/mail-2.7.1/lib/mail/mail.rb:235:in `block in inform_interceptors'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/mail-2.7.1/lib/mail/mail.rb:234:in `each'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/mail-2.7.1/lib/mail/mail.rb:234:in `inform_interceptors'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/mail-2.7.1/lib/mail/message.rb:248:in `inform_interceptors'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/mail-2.7.1/lib/mail/message.rb:258:in `deliver'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/actionmailer-7.0.1/lib/action_mailer/message_delivery.rb:119:in `block in deliver_now'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/actionmailer-7.0.1/lib/action_mailer/rescuable.rb:17:in `handle_exceptions'
	from /.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/actionmailer-7.0.1/lib/action_mailer/message_delivery.rb:118:in `deliver_now'

Workaround

Changing .first to .to_a.first on this line in premailer-rails allows me to work around this issue. I haven't deployed this so I can only confirm it works in development and with my tests.

[aaronjensen]

@agrberg I have the exact same circumstances. The same fixes it for me. Though it is called without a block initially, I believe that this line ends up recalling the same method with a block.

This seems like a Ruby bug to me or some breaking change I don't know about.

[aaronjensen]

Here is a repro:

https://github.com/aaronjensen/concurrent-repro

Install gems, then run bin/rails c and do:

::Rails.application.assets_manifest.find_sources("application.css").first

[agrberg]

I'm still tracking this down in a few spare moments. Via pry-byebug I was able to narrow the cause a bit further. SafeTaskExecutor in Promise#realize is intended to return a triple. However in our case at least both success and reason are nil. This is why there is no method exception on the reason object.

In cases where this is successful, success is true, value is set, and reason is nil. My hunch is that success is intended to be false when a proper reason exists vs. encountering this case where success is the only other falsey value, nil. I'll report back if I have any more luck!

[aaronjensen]

@agrberg Right, because for some reason the begin block at

concurrent-ruby/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb

Lines 23 to 29 in 52c08fc

	begin
	value = @task.call(*args)
	success = true
	rescue @exception_class => ex
	reason = ex
	success = false
	end

exits without recording an exception or getting to line 24.

The furthest I see things going is https://github.com/rails/sprockets/blob/v4.0.2/lib/sprockets/manifest.rb#L143 and then next thing I know it's returning from SafeTaskExecutor#execute.

[agrberg]

I found the same thing! I disabled the source of the error by changing this line to reason&.exception(*args) and now get a clearer error of Premailer::Rails::CSSHelper::FileNotFound specifically:

/.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/premailer-rails-1.11.1/lib/premailer/rails/css_helper.rb:51:in `load_css': File with URL "/assets/email.debug-#{long_hash}.css" could not be loaded by any strategy. (Premailer::Rails::CSSHelper::FileNotFound)

My hunch is that there is a race condition between this file being requested/accessed as when I "slow" the process down with my workaround, the information is found and everything works.

[aaronjensen]

I believe that error only happens because of the thing I described. It is not the actual error as far as I can tell, rather the natural consequence of the asset loader returning nil rather than an asset.

[aaronjensen]

I narrowed the repro down to just concurrent-ruby: https://github.com/aaronjensen/concurrent-repro

require "concurrent-ruby"

def sub
  yield
end

def run
  return to_enum(__method__) unless block_given?

  executor = :fast
  # This works
  # executor = :immediate

  promises = [
    Concurrent::Promise.execute(executor: executor) do
      # This version fails on Ruby 3.0.3 as well
      # yield "some-value"

      # This version only fails in Ruby 3.1.0
      sub do |value|
        yield "some-value"
      end
    end
  ]
  promises.each(&:wait!)

  nil
end

# This does not work
run.first

# This works
# run.to_a.first

puts "It worked"

[aaronjensen]

This was introduced by this commit:

938e027cdf019ff2cb6ee8a7229e6d9a4d8fc953 is the first bad commit
commit 938e027cdf019ff2cb6ee8a7229e6d9a4d8fc953
Author: Aaron Patterson <tenderlove@ruby-lang.org>
Date:   Tue Jan 26 15:49:21 2021 -0800

    Eliminate useless catch tables and nops from lambdas

cc @tenderlove

[aaronjensen]

It's seeming like this is a bug in concurrent-ruby, which does not account for the local jump that happens when an enumerator breaks when first is used. I'll look into seeing how I might fix it, but it would be great for one of the maintainers to chime in if they are available.

[aaronjensen]

#932 should fix it

[chrisseaton]

Thanks will have to find some time to get my head around the issue and the PR.

[agrberg]

Awesome job debugging this dude! 🙇