Skip to content

Builds two gems from one repo. #205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Jan 25, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
e597e04
Builds two gems from one repo. Specs and docs need updated.
jdantonio Dec 10, 2014
7fc55ec
Fixed broken CAtomic specs.
jdantonio Dec 11, 2014
be25542
Updated README with information regarding concurrent-ruby-ext
jdantonio Dec 11, 2014
011851d
Fixed bug with CAtomic erroneously being defined in pure-Ruby gem.
jdantonio Dec 14, 2014
57110e9
Extension version number now a constant within lib
jdantonio Dec 14, 2014
cc75945
Created specs to test the various gem builds.
jdantonio Dec 14, 2014
e7011f3
Improved build test runner.
jdantonio Dec 15, 2014
d742d08
Build tests now install RSpec when running tests.
jdantonio Dec 15, 2014
5aa707a
Updated build tests to support Windows.
jdantonio Dec 24, 2014
db81830
Core and extension gems now have same version number.
jdantonio Dec 29, 2014
0770644
Added note for gem developers to the README.
jdantonio Dec 29, 2014
0aa9b01
Fixed code samples in README
jdantonio Dec 29, 2014
d5d0c06
Updated benchmark scripts to check for C extensions.
jdantonio Dec 29, 2014
d3d9ab4
Fixed version number in build test runner.
jdantonio Dec 29, 2014
3369e46
Updated extensions and build tasks based on Heroku testing.
jdantonio Dec 30, 2014
1a2377e
Better platform detection in Rake compile and build tasks.
jdantonio Dec 30, 2014
20f0e44
Fixed cross-compile build problems in Rakefile.
jdantonio Dec 30, 2014
37e5dc3
Added build information to the README.
jdantonio Dec 30, 2014
495113c
Fixed bad Java compile task.
jdantonio Jan 15, 2015
7d48463
Better warning when using pure-Ruby build under JRuby.
jdantonio Jan 15, 2015
3877635
Created build tests for CAtomic and JavaAtomic.
jdantonio Jan 16, 2015
1795faa
Fixed build tests for JRuby.
jdantonio Jan 25, 2015
12ba3a8
Fixed bug with Windows pre-compiled extension loading.
jdantonio Jan 25, 2015
6a17ca0
Fixed tests for JavaAtomic.
jdantonio Jan 25, 2015
372a68f
Fixed build tests on Windows.
jdantonio Jan 25, 2015
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,4 @@ matrix:
- rvm: jruby-head
- rvm: 1.9.3

script: "bundle exec rake compile && bundle exec rspec --color --backtrace --tag ~unfinished --seed 1 --format documentation ./spec"
script: "rake compile && bundle exec rspec --color --backtrace --tag ~unfinished --seed 1 --format documentation ./spec"
3 changes: 2 additions & 1 deletion Gemfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
source 'https://rubygems.org'

gemspec
gemspec name: 'concurrent-ruby'

group :development do
gem 'rake', '~> 10.3.2'
gem 'rake-compiler', '~> 0.9.2'
gem 'gem-compiler', '~> 0.3.0'
end

group :testing do
Expand Down
130 changes: 90 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,8 @@

### Supported Ruby versions

MRI 1.9.3, 2.0, 2.1, JRuby (1.9 mode), and Rubinius 2.x are supported.
Although native code is used for performance optimizations on some platforms, all functionality
is available in pure Ruby. This gem should be fully compatible with any interpreter that is
compliant with Ruby 1.9.3 or newer.
MRI 1.9.3, 2.0, 2.1, 2.2, JRuby (1.9 mode), and Rubinius 2.x are supported.
This gem should be fully compatible with any interpreter that is compliant with Ruby 1.9.3 or newer.

## Features & Documentation

Expand All @@ -62,6 +60,7 @@ This library contains a variety of concurrency abstractions at high and low leve
* [Promise](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/Promise.html): Similar to Futures, with more features.
* [ScheduledTask](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/ScheduledTask.html): Like a Future scheduled for a specific future time.
* [TimerTask](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/TimerTask.html): A Thread that periodically wakes up to perform work at regular intervals.
* [Channel](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/Channel.html): Communicating Sequential Processes (CSP).

### Java-inspired ThreadPools and other executors

Expand Down Expand Up @@ -98,28 +97,45 @@ Lower-level abstractions mainly used as building blocks.
* [thread-local variables](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/ThreadLocalVar.html)
* [software transactional memory](http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/TVar.html) (TVar)

## Installing and Building
## Usage

This gem includes several platform-specific optimizations. To reduce the possibility of
compilation errors, we provide pre-compiled gem packages for several platforms as well
as a pure-Ruby build. Installing the gem should be no different than installing any other
Rubygems-hosted gem. Rubygems will automatically detect your platform and install the
appropriate pre-compiled build. You should never see Rubygems attempt to compile the gem
on installation. Additionally, to ensure compatability with the largest possible number
of Ruby interpreters, the C extensions will *never* load under any Ruby other than MRI,
even when installed.
All abstractions within this gem can be loaded simply by requiring it:

The following gem builds will be built at every release:
```ruby
require 'concurrent'
```

To reduce the amount of code loaded at runtime, subsets of this gem can be required:

* concurrent-ruby-x.y.z.gem (pure Ruby)
* concurrent-ruby-x.y.z-java.gem (JRuby)
* concurrent-ruby-x.y.z-x86-linux.gem (Linux 32-bit)
* concurrent-ruby-x.y.z-x86_64-linux.gem (Linux 64-bit)
* concurrent-ruby-x.y.z-x86-mingw32.gem (Windows 32-bit)
* concurrent-ruby-x.y.z-x64-mingw32.gem (Windows 64-bit)
* concurrent-ruby-x.y.z-x86-solaris-2.11.gem (Solaris)
```ruby
require 'concurrent' # everything

# groups

require 'concurrent/actor' # Concurrent::Actor and supporting code
require 'concurrent/atomics' # atomic and thread synchronization classes
require 'concurrent/channels' # Concurrent::Channel and supporting code
require 'concurrent/executors' # Thread pools and other executors
require 'concurrent/utilities' # utility methods such as processor count and timers

# individual abstractions

require 'concurrent/agent' # Concurrent::Agent
require 'concurrent/async' # Concurrent::Async
require 'concurrent/atomic' # Concurrent::Atomic (formerly the `atomic` gem)
require 'concurrent/dataflow' # Concurrent::dataflow
require 'concurrent/delay' # Concurrent::Delay
require 'concurrent/exchanger' # Concurrent::Exchanger
require 'concurrent/future' # Concurrent::Future
require 'concurrent/ivar' # Concurrent::IVar
require 'concurrent/mvar' # Concurrent::MVar
require 'concurrent/promise' # Concurrent::Promise
require 'concurrent/scheduled_task' # Concurrent::ScheduledTask
require 'concurrent/timer_task' # Concurrent::TimerTask
require 'concurrent/tvar' # Concurrent::TVar
```

### Installing
## Installation

```shell
gem install concurrent-ruby
Expand All @@ -133,31 +149,64 @@ gem 'concurrent-ruby'

and run `bundle install` from your shell.

### Building
### C Extensions for MRI

Because we provide pre-compiled gem builds, users should never need to build the gem manually.
The build process for this gem is completely automated using open source tools. All of
the automation components are available in the [ruby-concurrency/rake-compiler-dev-box](https://github.com/ruby-concurrency/rake-compiler-dev-box)
GitHub repository.
Potential performance improvements may be achieved under MRI by installing optional C extensions.
To minimize installation errors the C extensions are available in the `concurrent-ruby-ext` extension
gem. The extension gem lists `concurrent-ruby` as a dependency so it is not necessary to install both.
Simply install the extension gen:

This gem will compile native C code under MRI and native Java code under JRuby. It is
also possible to build a pure-Ruby version. All builds have identical functionality.
The only difference is performance. Additionally, pure-Ruby classes are always available,
even when using the native optimizations. Please see the [documentation](http://ruby-concurrency.github.io/concurrent-ruby/)
for more details.
```ruby
gem install concurrent-ruby-ext
```

To build and package the gem using MRI or JRuby, install the necessary build dependencies and run:
or add the following line to Gemfile:

```shell
bundle exec rake compile
bundle exec rake build
```ruby
gem 'concurrent-ruby-ext'
```

and run `bundle install` from your shell.

In code it is only necessary to

```ruby
require 'concurrent'
```

To build and package a pure-Ruby gem, on *any* platform and interpreter
(including MRI and JRuby), run:
The `concurrent-ruby` gem will automatically detect the presence of the `concurrent-ruby-ext` gem
and load the appropriate C extensions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like:

For gem developers

No gems should depend on concurrent-ruby-ext. Otherwise C extensions are forced to users of your dependent gem. Best to depend on concurrent-ruby and let users to decide if they want C extensions.

could help gem developers.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

```shell
BUILD_PURE_RUBY='true' bundle exec rake build
#### Note For gem developers

No gems should depend on `concurrent-ruby-ext`. Doing so will force C extensions on your users.
The best practice is to depend on `concurrent-ruby` and let users to decide if they want C extensions.

### Building

All published versions of this gem (core, extension, and several platform-specific packages) are compiled,
packaged, tested, and published using an open, [automated process](https://github.com/ruby-concurrency/rake-compiler-dev-box).
This process can also be used to create pre-compiled binaries of the extension gem for virtally
any platform. *Documentation is forthcoming...*

```
*MRI only*
rake build:native # Build concurrent-ruby-ext-<version>-<platform>.gem into the pkg directory
rake compile:extension # Compile extension

*JRuby only*
rake build # Build JRuby-specific core gem (alias for `build:core`)
rake build:core # Build concurrent-ruby-<version>-java.gem into the pkg directory

*All except JRuby*
rake build # Build core and extension gems
rake build:core # Build concurrent-ruby-<version>.gem into the pkg directory
rake build:ext # Build concurrent-ruby-ext-<version>.gem into the pkg directory

*All*
rake clean # Remove any temporary products
rake clobber # Remove any generated file
rake compile # Compile all the extensions
```

## Maintainers
Expand All @@ -167,6 +216,7 @@ BUILD_PURE_RUBY='true' bundle exec rake build
* [Chris Seaton](https://github.com/chrisseaton)
* [Lucas Allan](https://github.com/lucasallan)
* [Petr Chalupa](https://github.com/pitr-ch)
* [Paweł Obrok](https://github.com/obrok)

### Contributing

Expand Down
98 changes: 77 additions & 21 deletions Rakefile
Original file line number Diff line number Diff line change
@@ -1,20 +1,32 @@
require 'bundler/gem_tasks'
require 'rake/extensiontask'
require 'rake/javaextensiontask'
#!/usr/bin/env rake

GEMSPEC = Gem::Specification.load('concurrent-ruby.gemspec')
EXTENSION_NAME = 'concurrent_ruby_ext'
require_relative './lib/extension_helper'

Bundler::GemHelper.install_tasks
## load the two gemspec files
CORE_GEMSPEC = Gem::Specification.load('concurrent-ruby.gemspec')
EXT_GEMSPEC = Gem::Specification.load('concurrent-ruby-ext.gemspec')

$:.push File.join(File.dirname(__FILE__), 'lib')
require 'extension_helper'
## constants used for compile/build tasks

GEM_NAME = 'concurrent-ruby'
EXTENSION_NAME = 'extension'
JAVA_EXT_NAME = 'concurrent_ruby_ext'

if Concurrent.jruby?
CORE_GEM = "#{GEM_NAME}-#{Concurrent::VERSION}-java.gem"
else
CORE_GEM = "#{GEM_NAME}-#{Concurrent::VERSION}.gem"
EXTENSION_GEM = "#{GEM_NAME}-ext-#{Concurrent::VERSION}.gem"
NATIVE_GEM = "#{GEM_NAME}-ext-#{Concurrent::VERSION}-#{Gem::Platform.new(RUBY_PLATFORM)}.gem"
end

## safely load all the rake tasks in the `tasks` directory

def safe_load(file)
begin
load file
rescue LoadError => ex
puts 'Error loading rake tasks, but will continue...'
puts "Error loading rake tasks from '#{file}' but will continue..."
puts ex.message
end
end
Expand All @@ -23,21 +35,24 @@ Dir.glob('tasks/**/*.rake').each do |rakefile|
safe_load rakefile
end

desc 'Run benchmarks'
task :bench do
exec 'ruby -Ilib -Iext examples/bench_atomic.rb'
end
if Concurrent.jruby?

if defined?(JRUBY_VERSION)
## create the compile task for the JRuby-specific gem
require 'rake/javaextensiontask'

Rake::JavaExtensionTask.new(EXTENSION_NAME, GEMSPEC) do |ext|
Rake::JavaExtensionTask.new(JAVA_EXT_NAME, CORE_GEMSPEC) do |ext|
ext.ext_dir = 'ext'
end

elsif Concurrent.allow_c_extensions?

Rake::ExtensionTask.new(EXTENSION_NAME, GEMSPEC) do |ext|
ext.ext_dir = "ext/#{EXTENSION_NAME}"
## create the compile tasks for the extension gem
require 'rake/extensiontask'

Rake::ExtensionTask.new(EXTENSION_NAME, EXT_GEMSPEC) do |ext|
ext.ext_dir = 'ext/concurrent'
ext.lib_dir = 'lib/concurrent'
ext.source_pattern = '*.{c,h}'
ext.cross_compile = true
ext.cross_platform = ['x86-mingw32', 'x64-mingw32']
end
Expand All @@ -59,20 +74,61 @@ elsif Concurrent.allow_c_extensions?
end
end
else
task :clean
## create an empty compile task
task :compile
task "compile:#{EXTENSION_NAME}"
end

Rake::Task[:clean].enhance do
task :clean do
rm_rf 'pkg/classes'
rm_rf 'tmp'
rm_rf 'lib/1.9'
rm_rf 'lib/2.0'
rm_f Dir.glob('./lib/*.jar')
rm_f Dir.glob('./**/*.so')
rm_f Dir.glob('./**/*.bundle')
rm_f Dir.glob('./lib/*.jar')
mkdir_p 'pkg'
end

## create build tasks tailored to current platform

namespace :build do

build_deps = [:clean]
build_deps << :compile if Concurrent.jruby?

desc "Build #{CORE_GEM} into the pkg directory"
task :core => build_deps do
sh "gem build #{CORE_GEMSPEC.name}.gemspec"
sh 'mv *.gem pkg/'
end

unless Concurrent.jruby?
desc "Build #{EXTENSION_GEM} into the pkg directory"
task :ext => [:clean] do
sh "gem build #{EXT_GEMSPEC.name}.gemspec"
sh 'mv *.gem pkg/'
end
end

if Concurrent.allow_c_extensions?
desc "Build #{NATIVE_GEM} into the pkg directory"
task :native do
sh "gem compile pkg/#{EXTENSION_GEM}"
sh 'mv *.gem pkg/'
end
end
end

if Concurrent.jruby?
desc 'Build JRuby-specific core gem (alias for `build:core`)'
task :build => ['build:core']
else
desc 'Build core and extension gems'
task :build => ['build:core', 'build:ext']
end

## the RSpec task that compiles extensions when available

begin
require 'rspec'
require 'rspec/core/rake_task'
Expand Down
Loading