Merge pull request #205 from ruby-concurrency/refactor/two-gems-one-repo

Builds two gems from one repo.

Author: jdantonio

.travis.yml

@@ -25,4 +25,4 @@ matrix:
     - rvm: jruby-head
     - rvm: 1.9.3
 
-script: "bundle exec rake compile && bundle exec rspec --color --backtrace --tag ~unfinished --seed 1 --format documentation ./spec"
+script: "rake compile && bundle exec rspec --color --backtrace --tag ~unfinished --seed 1 --format documentation ./spec"

Gemfile

@@ -1,10 +1,11 @@
 source 'https://rubygems.org'
 
-gemspec
+gemspec name: 'concurrent-ruby'
 
 group :development do
   gem 'rake', '~> 10.3.2'
   gem 'rake-compiler', '~> 0.9.2'
+  gem 'gem-compiler', '~> 0.3.0'
 end
 
 group :testing do

README.md

@@ -37,10 +37,8 @@
 
 ### Supported Ruby versions
 
-MRI 1.9.3, 2.0, 2.1, JRuby (1.9 mode), and Rubinius 2.x are supported.
-Although native code is used for performance optimizations on some platforms, all functionality
-is available in pure Ruby. This gem should be fully compatible with any interpreter that is
-compliant with Ruby 1.9.3 or newer.
+MRI 1.9.3, 2.0, 2.1, 2.2, JRuby (1.9 mode), and Rubinius 2.x are supported.
+This gem should be fully compatible with any interpreter that is compliant with Ruby 1.9.3 or newer.
 
 ## Features & Documentation
 
@@ -62,6 +60,7 @@ This library contains a variety of concurrency abstractions at high and low leve
 * [Promise](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/Promise.html): Similar to Futures, with more features.
 * [ScheduledTask](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/ScheduledTask.html): Like a Future scheduled for a specific future time.
 * [TimerTask](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/TimerTask.html): A Thread that periodically wakes up to perform work at regular intervals. 
+* [Channel](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/Channel.html): Communicating Sequential Processes (CSP). 
 
 ### Java-inspired ThreadPools and other executors
 
@@ -98,28 +97,45 @@ Lower-level abstractions mainly used as building blocks.
 * [thread-local variables](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/ThreadLocalVar.html)
 * [software transactional memory](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/TVar.html) (TVar)
 
-## Installing and Building
+## Usage
 
-This gem includes several platform-specific optimizations. To reduce the possibility of
-compilation errors, we provide pre-compiled gem packages for several platforms as well
-as a pure-Ruby build. Installing the gem should be no different than installing any other
-Rubygems-hosted gem. Rubygems will automatically detect your platform and install the
-appropriate pre-compiled build. You should never see Rubygems attempt to compile the gem
-on installation. Additionally, to ensure compatability with the largest possible number
-of Ruby interpreters, the C extensions will *never* load under any Ruby other than MRI,
-even when installed.
+All abstractions within this gem can be loaded simply by requiring it:
 
-The following gem builds will be built at every release:
+```ruby
+require 'concurrent'
+```
+
+To reduce the amount of code loaded at runtime, subsets of this gem can be required:
 
-* concurrent-ruby-x.y.z.gem (pure Ruby)
-* concurrent-ruby-x.y.z-java.gem (JRuby)
-* concurrent-ruby-x.y.z-x86-linux.gem (Linux 32-bit)
-* concurrent-ruby-x.y.z-x86_64-linux.gem (Linux 64-bit)
-* concurrent-ruby-x.y.z-x86-mingw32.gem (Windows 32-bit)
-* concurrent-ruby-x.y.z-x64-mingw32.gem (Windows 64-bit)
-* concurrent-ruby-x.y.z-x86-solaris-2.11.gem (Solaris)
+```ruby
+require 'concurrent'                # everything
+
+# groups
+
+require 'concurrent/actor'          # Concurrent::Actor and supporting code
+require 'concurrent/atomics'        # atomic and thread synchronization classes
+require 'concurrent/channels'       # Concurrent::Channel and supporting code
+require 'concurrent/executors'      # Thread pools and other executors
+require 'concurrent/utilities'      # utility methods such as processor count and timers
+
+# individual abstractions
+
+require 'concurrent/agent'          # Concurrent::Agent
+require 'concurrent/async'          # Concurrent::Async
+require 'concurrent/atomic'         # Concurrent::Atomic (formerly the `atomic` gem)
+require 'concurrent/dataflow'       # Concurrent::dataflow
+require 'concurrent/delay'          # Concurrent::Delay
+require 'concurrent/exchanger'      # Concurrent::Exchanger
+require 'concurrent/future'         # Concurrent::Future
+require 'concurrent/ivar'           # Concurrent::IVar
+require 'concurrent/mvar'           # Concurrent::MVar
+require 'concurrent/promise'        # Concurrent::Promise
+require 'concurrent/scheduled_task' # Concurrent::ScheduledTask
+require 'concurrent/timer_task'     # Concurrent::TimerTask
+require 'concurrent/tvar'           # Concurrent::TVar
+```
 
-### Installing
+## Installation
 
 ```shell
 gem install concurrent-ruby
@@ -133,31 +149,64 @@ gem 'concurrent-ruby'
 
 and run `bundle install` from your shell.
 
-### Building
+### C Extensions for MRI
 
-Because we provide pre-compiled gem builds, users should never need to build the gem manually.
-The build process for this gem is completely automated using open source tools. All of
-the automation components are available in the [ruby-concurrency/rake-compiler-dev-box](https://github.com/ruby-concurrency/rake-compiler-dev-box)
-GitHub repository.
+Potential performance improvements may be achieved under MRI by installing optional C extensions.
+To minimize installation errors the C extensions are available in the `concurrent-ruby-ext` extension
+gem. The extension gem lists `concurrent-ruby` as a dependency so it is not necessary to install both.
+Simply install the extension gen:
 
-This gem will compile native C code under MRI and native Java code under JRuby. It is
-also possible to build a pure-Ruby version. All builds have identical functionality.
-The only difference is performance. Additionally, pure-Ruby classes are always available,
-even when using the native optimizations. Please see the [documentation](http://ruby-concurrency.github.io/concurrent-ruby/)
-for more details.
+```ruby
+gem install concurrent-ruby-ext
+```
 
-To build and package the gem using MRI or JRuby, install the necessary build dependencies and run:
+or add the following line to Gemfile:
 
-```shell
-bundle exec rake compile
-bundle exec rake build
+```ruby
+gem 'concurrent-ruby-ext'
+```
+
+and run `bundle install` from your shell.
+
+In code it is only necessary to
+
+```ruby
+require 'concurrent'
 ```
 
-To build and package a pure-Ruby gem, on *any* platform and interpreter
-(including MRI and JRuby), run:
+The `concurrent-ruby` gem will automatically detect the presence of the `concurrent-ruby-ext` gem
+and load the appropriate C extensions.
 
-```shell
-BUILD_PURE_RUBY='true' bundle exec rake build
+#### Note For gem developers
+
+No gems should depend on `concurrent-ruby-ext`. Doing so will force C extensions on your users.
+The best practice is to depend on `concurrent-ruby` and let users to decide if they want C extensions.
+
+### Building
+
+All published versions of this gem (core, extension, and several platform-specific packages) are compiled,
+packaged, tested, and published using an open, [automated process](https://github.com/ruby-concurrency/rake-compiler-dev-box).
+This process can also be used to create pre-compiled binaries of the extension gem for virtally
+any platform. *Documentation is forthcoming...*
+
+```
+*MRI only*
+rake build:native       # Build concurrent-ruby-ext-<version>-<platform>.gem into the pkg directory
+rake compile:extension  # Compile extension
+
+*JRuby only*
+rake build              # Build JRuby-specific core gem (alias for `build:core`)
+rake build:core         # Build concurrent-ruby-<version>-java.gem into the pkg directory
+
+*All except JRuby*
+rake build              # Build core and extension gems
+rake build:core         # Build concurrent-ruby-<version>.gem into the pkg directory
+rake build:ext          # Build concurrent-ruby-ext-<version>.gem into the pkg directory
+
+*All*
+rake clean              # Remove any temporary products
+rake clobber            # Remove any generated file
+rake compile            # Compile all the extensions
 ```
 
 ## Maintainers
@@ -167,6 +216,7 @@ BUILD_PURE_RUBY='true' bundle exec rake build
 * [Chris Seaton](https://github.com/chrisseaton)
 * [Lucas Allan](https://github.com/lucasallan)
 * [Petr Chalupa](https://github.com/pitr-ch)
+* [Paweł Obrok](https://github.com/obrok)
 
 ### Contributing
 

Rakefile

@@ -1,20 +1,32 @@
-require 'bundler/gem_tasks'
-require 'rake/extensiontask'
-require 'rake/javaextensiontask'
+#!/usr/bin/env rake
 
-GEMSPEC = Gem::Specification.load('concurrent-ruby.gemspec')
-EXTENSION_NAME = 'concurrent_ruby_ext'
+require_relative './lib/extension_helper'
 
-Bundler::GemHelper.install_tasks
+## load the two gemspec files
+CORE_GEMSPEC = Gem::Specification.load('concurrent-ruby.gemspec')
+EXT_GEMSPEC = Gem::Specification.load('concurrent-ruby-ext.gemspec')
 
-$:.push File.join(File.dirname(__FILE__), 'lib')
-require 'extension_helper'
+## constants used for compile/build tasks
+
+GEM_NAME = 'concurrent-ruby'
+EXTENSION_NAME = 'extension'
+JAVA_EXT_NAME = 'concurrent_ruby_ext'
+
+if Concurrent.jruby?
+  CORE_GEM = "#{GEM_NAME}-#{Concurrent::VERSION}-java.gem"
+else
+  CORE_GEM = "#{GEM_NAME}-#{Concurrent::VERSION}.gem"
+  EXTENSION_GEM = "#{GEM_NAME}-ext-#{Concurrent::VERSION}.gem"
+  NATIVE_GEM = "#{GEM_NAME}-ext-#{Concurrent::VERSION}-#{Gem::Platform.new(RUBY_PLATFORM)}.gem"
+end
+
+## safely load all the rake tasks in the `tasks` directory
 
 def safe_load(file)
   begin
     load file
   rescue LoadError => ex
-    puts 'Error loading rake tasks, but will continue...'
+    puts "Error loading rake tasks from '#{file}' but will continue..."
     puts ex.message
   end
 end
@@ -23,21 +35,24 @@ Dir.glob('tasks/**/*.rake').each do |rakefile|
   safe_load rakefile
 end
 
-desc 'Run benchmarks'
-task :bench do
-  exec 'ruby -Ilib -Iext examples/bench_atomic.rb'
-end
+if Concurrent.jruby?
 
-if defined?(JRUBY_VERSION)
+  ## create the compile task for the JRuby-specific gem
+  require 'rake/javaextensiontask'
 
-  Rake::JavaExtensionTask.new(EXTENSION_NAME, GEMSPEC) do |ext|
+  Rake::JavaExtensionTask.new(JAVA_EXT_NAME, CORE_GEMSPEC) do |ext|
     ext.ext_dir = 'ext'
   end
 
 elsif Concurrent.allow_c_extensions?
 
-  Rake::ExtensionTask.new(EXTENSION_NAME, GEMSPEC) do |ext|
-    ext.ext_dir = "ext/#{EXTENSION_NAME}"
+  ## create the compile tasks for the extension gem
+  require 'rake/extensiontask'
+
+  Rake::ExtensionTask.new(EXTENSION_NAME, EXT_GEMSPEC) do |ext|
+    ext.ext_dir = 'ext/concurrent'
+    ext.lib_dir = 'lib/concurrent'
+    ext.source_pattern = '*.{c,h}'
     ext.cross_compile = true
     ext.cross_platform = ['x86-mingw32', 'x64-mingw32']
   end
@@ -59,20 +74,61 @@ elsif Concurrent.allow_c_extensions?
     end
   end
 else
-  task :clean
+  ## create an empty compile task
   task :compile
-  task "compile:#{EXTENSION_NAME}"
 end
 
-Rake::Task[:clean].enhance do
+task :clean do
   rm_rf 'pkg/classes'
   rm_rf 'tmp'
   rm_rf 'lib/1.9'
   rm_rf 'lib/2.0'
-  rm_f Dir.glob('./lib/*.jar')
+  rm_f Dir.glob('./**/*.so')
   rm_f Dir.glob('./**/*.bundle')
+  rm_f Dir.glob('./lib/*.jar')
+  mkdir_p 'pkg'
 end
 
+## create build tasks tailored to current platform
+
+namespace :build do
+
+  build_deps = [:clean]
+  build_deps << :compile if Concurrent.jruby?
+
+  desc "Build #{CORE_GEM} into the pkg directory"
+  task :core => build_deps do
+    sh "gem build #{CORE_GEMSPEC.name}.gemspec"
+    sh 'mv *.gem pkg/'
+  end
+
+  unless Concurrent.jruby?
+    desc "Build #{EXTENSION_GEM} into the pkg directory"
+    task :ext => [:clean] do
+      sh "gem build #{EXT_GEMSPEC.name}.gemspec"
+      sh 'mv *.gem pkg/'
+    end
+  end
+
+  if Concurrent.allow_c_extensions?
+    desc "Build #{NATIVE_GEM} into the pkg directory"
+    task :native do
+      sh "gem compile pkg/#{EXTENSION_GEM}"
+      sh 'mv *.gem pkg/'
+    end
+  end
+end
+
+if Concurrent.jruby?
+  desc 'Build JRuby-specific core gem (alias for `build:core`)'
+  task :build => ['build:core']
+else
+  desc 'Build core and extension gems'
+  task :build => ['build:core', 'build:ext']
+end
+
+## the RSpec task that compiles extensions when available
+
 begin
   require 'rspec'
   require 'rspec/core/rake_task'