undefined method `time' on nil:NilClass. (NoMethodError)

[asppsa]

Hi there,

I'm seeing error messages like the following while stress-testing a server I'm making that uses concurrent-ruby.

[2014-10-25T15:39:16.169601 #23662] DEBUG -- : undefined method `time' on nil:NilClass. (NoMethodError)
kernel/delta/kernel.rb:78:in `time (method_missing)'
/home/asp/.rvm/gems/rbx-2.2.10/bundler/gems/concurrent-ruby-041196d6de89/lib/concurrent/executor/timer_set.rb:130:in `process_tasks'
kernel/common/kernel.rb:460:in `loop'
/home/asp/.rvm/gems/rbx-2.2.10/bundler/gems/concurrent-ruby-041196d6de89/lib/concurrent/executor/timer_set.rb:126:in `process_tasks'
kernel/common/method.rb:71:in `call'
kernel/bootstrap/proc.rb:18:in `call'
/home/asp/.rvm/gems/rbx-2.2.10/bundler/gems/concurrent-ruby-041196d6de89/lib/concurrent/executor/ruby_single_thread_executor.rb:65:in `work'
kernel/common/kernel.rb:460:in `loop'
/home/asp/.rvm/gems/rbx-2.2.10/bundler/gems/concurrent-ruby-041196d6de89/lib/concurrent/executor/ruby_single_thread_executor.rb:61:in `work'
/home/asp/.rvm/gems/rbx-2.2.10/bundler/gems/concurrent-ruby-041196d6de89/lib/concurrent/executor/ruby_single_thread_executor.rb:55:in `new_worker_thread'
kernel/bootstrap/proc.rb:20:in `call'
kernel/bootstrap/thread.rb:391:in `__run__'

The issue appears in 0.7.0 as well as with a checkout of the latest code from git. Under Rubinius I get error messages; under jruby things just seem to freeze, and I haven't tried MRI. These errors (or lack of errors for jruby) seem to coincide with all actors in the system ceasing to respond to messages.

See this gist for example which reproduces the problem, along with sample output from running in Rubinius on my Linux system. I'm not really sure what part of this is making things break, but it seems to be a combination of high load, timer tasks and terminated actors with children. I don't think I'm doing anything "illegal" in the example, but I may well be wrong!

Thanks for all your work on this gem. This issue aside, I'm finding these concurrency tools to be a huge game-changer for programming in Ruby.

[jdantonio]

@asppsa Thank you for both the bug report and your kind words. We're glad you find our gem useful! Unfortunately I won't be able to look at this until next week. I'm currently visiting Central Europe on vacation (I live in the US; today I am in Bucharest, Romania; tomorrow we head to Brasov). I apologize for the delay. I plan to create a new release before RubyConf in November. I will try my best to find the problem before then.

[pitr-ch]

Hello @asppsa, thanks! I was able to reproduce it under JRuby, so far my best guess is combination of 2 problems:

• starved thread-pool, a blocking tasks are getting to task_pool which is not allowed by contract
• race condition on the Task object in TimerSet, it's synchronized only when pushed to queue but not when reading in the worker loop.

I'll came back to it later and investigate more.

[pitr-ch]

When you add

Concurrent.configuration.instance_eval do
  @global_timer_set.reconfigure { Concurrent::TimerSet.new executor: Concurrent.configuration.global_operation_pool }
end

Concurrent.configuration.global_operation_pool = Concurrent::ThreadPoolExecutor.new(
    min_threads:     [2, Concurrent.processor_count].max,
    max_threads:     [2, Concurrent.processor_count].max,
    idletime:        10 * 60, # 10 minutes
    max_queue:       0, # unlimited
    overflow_policy: :abort # raise an exception
)

before Actor1 definition:

• it moves TimerTask execution from task-pool to operation-pool since it has blocking call in it (ask!)
• it changes operation pool configuration with unlimited queue of jobs.
• Note: promises are already on operation pool

This changes fixed it for me under JRuby.

[pitr-ch]

@asppsa please try #179 if it works for you too on rbx.

[asppsa]

Hi there,

Yes, this seems to fix the issue with the error message -- I no longer see it when running my test or when working on my real project using rbx. It looks like my project must be doing something similar to what you uncovered above -- i.e. starving the task pool somehow, so thank you very much for the diagnosis!

[jdantonio]

@pitr-ch Thank you for taking care of this. I return to the States later today (currently in Munich airport awaiting my flight home) so this weekend I should be able to review the recent activity.