RxJava 2.0

[benjchristensen]

Looking towards the future this issue is for discussing what RxJava 2.0 might look like.

Current thinking (as of early 2015) is to target Java 9 and leverage JDK/JVM functionality added in Java 8/9 such as lambdas, value types (hopefully in Java 9), Stream and CompletableFuture.

It would also make sense to align more closely with Reactive Streams (https://github.com/reactive-streams/reactive-streams) since RxJava was involved in defining it. This would eliminate the need for a bridge (https://github.com/ReactiveX/RxJavaReactiveStreams).

RxJava 2.0 must be namespaced separately from 1.0 since RxJava 1.0 is committed to never breaking. This is essential in order for RxJava to be usable similar to java.util collections.

The proposed namespacing is:

Java Package: io.reactivex.* (changed from rx.*)
Maven GroupID: io.reactivex (no change)
Maven ArtifactId: rxjava2 (changed from rxjava)

Issues for 2.0: https://github.com/ReactiveX/RxJava/milestones/2.0

[smaldini]

We have a similar issue for the future of Reactor, and after spending months implementing a reasonable scope of Rx (but native Reactive Stream Publisher/Subscriber) I feel we can do better for the JVM community !

That essentially calls for some convergence and reduce the scope of Reactor and contribute things to ReactiveX, depends on it and continue there. The fact we follow the contract and the naming alignment is a first step IMO. As you know some operations will probably be exception to the spec.

What do we have mainly in reactor-core and how it compares to Rx:

• Environment : very light version of Rx context (plugins etc) scope, registry of Dispatchers and CircuitMetrics (TBA), builder for Dispatchers
• Dispatcher : equivalent of Scheduler, we offer simple MPSC dispatcher (based on JCTools), but the star dispatcher is the RingBufferDispatcher which offers nice perfs and lock free dispatching.
• JDK 8 port for functions, consumers etc, Timer, Tuple (reactor.fn)
• Event, Selector, EventBus, Observable (reactor.bus) : All facilities for fully decoupled signal passing, with support for headers and fine grained routing
• Async utils (reactor.core) : ForkJoin stuff, Batch RingBuffer processing etc.
• Streams, Promises and Stream, Promise, BiStreams (reactor.rx) : very equivalent to RxJava, less lengthy API wise, but with some different operations and implementation based on Reactive Streams and reactor.fn (ultimately java 8). Some operations are related to EventBus or Selector, some to Timer. The core backpressure mech is using RS and we offer some alias for requesting (unbounded or not)
  • A note on Action (reactor.rx.action), it combines both Subject and Operator (to become a Processor). We lift these actions on subscribe for unicasting unless root publisher is also an Action (offering then a pub/sub behavior at this point). Actions are like actors with an assigned dispatcher (can be synchronous). They track demand on their upstream and forward data using downstream Subscription.
• Codec, ChronicleStream, IOStreams, Buffer, PersistentQueue(reactor.io) : RS based implementation for event driven map (persisted and/or replicated with Chronicle), queues, byte utils, codec for signal conversion...

So the gap is in the implementation of a specific dispatcher, metrics for stream/bus, (async utils?), io operations, key/value operations and mainly the native work with RS/JDK8-like interface. We can certainly help transforming rxjava 2.0 into a standard RS impl and try to provide some sort of convergence for the stuff we get rid off. Ultimately that would be great if sticking to naming from rx (and we have experimented switching between RxJava and Reactor), the transition won't be painful for the users. However this will sometimes call for behavior change due to the spec but not too much, coming from rxjava 1.x at least.

I think we both have a mission to educate and propagate the practice both in Rx and Reactive Stream and that's great as our respective users will converge naturally 👍

[abersnaze]

It would also be interesting to see how we could leverage Java 8's pluggable types. We could potentially validate at compile time things that we currently can't detect until run time. Things like back pressure support or that Subscribers are implemented correctly.

[benjchristensen]

The Reactive Stream types have been proposed for Java 9: http://cs.oswego.edu/pipermail/concurrency-interest/2015-January/013641.html

These are part of what we'd want to target for RxJava 2.0 for Java 9.

[akarnokd]

As with Rx.NET, getting the base interfaces into Java is a good start. But I'm not sure what services/operators Java 9 wants to provide.

Since Streams are single use only, working with them has to be either single subscriber or multicast all the time.

CompletableFuture in itself is not that interesting since our Schedulers and workers and operators already do the heavy lifting; I see only minimal interop possibility, such as seeing an AsyncSubject as a CompletableFuture and quit RxJava at that spot.

[benjchristensen]

I'm not sure what services/operators Java 9 wants to provide.

Yes, I'm not suggesting Java 9 will negate the need or desire for RxJava and its many operators, but that we want to target Java 9 and support the types and interop it will allow.

I see only minimal interop possibility

It's the same type of interop that would be permitted with Reactive Streams, except now standardized in the JDK. Not a big deal when only within the RxJava world, but quite useful when that's not the reality.

Examples include how there are bindings and interop between Vert.x, Ratpack, Jersey, Akka, Reactor, etc.

[daschl]

I totally agree with @benjchristensen. for example in "our" case, we don't have much control of where people deploy the SDK. If they are in a spring world and they use Reactor, with Reactive Streams (or any other interop), the have a way to transfer the powerful concepts into "their" world without loosing semantics or have to switch to blocking operations.

Also on the pragmatic side: the less code RxJava has to maintain and the oracle folks do instead, the better. This also means greater adoption in all places, and people are going to build stuff we don't even know about yet.

[benjchristensen]

My view of the Java 9 world is that RxJava stops needing to define the foundational interfaces and semantics as JDK 9 will now include them, and it becomes more focused on being extensions to Java 9, as per its "Reactive Extensions" name, and providing a library of operators that can be composed and used when building reactive applications.

[akarnokd]

Now we can use Runnables, Consumers and Predicates directly. However, it could take quite some time that IDEs start to support enhanced volatiles and we certainly lose Unsafe. I've asked about operator support in the concurrency-interest list and it seems there won't be any default implementation or set of operators at all, so no default fluent JDK API. The drawback is that we'll end up with Observable.from(j.u.c.Flow.Producer) all over the place and no unsafeSubscribe.

[benjchristensen]

Capturing some further thoughts on this ... a major area of focus for RxJava 2.0 is to improve performance in areas where we have learned that the design impedes us.

Unsubscribe Concurrency

One such area is how concurrent unsubscribes require the use of volatile or synchronization in many places we'd rather not. Supporting a potential concurrent unsubscribe necessitates paying the cost for every emission. I have seen this cost millions of ops/second. I would like us to explore only allowing unsubscribes "in the line of fire" and making concurrent unsubscribes something that is opt-in via an operator, such as using takeUntil which will pay the synchronization cost only there when used, rather than all the time such as Subscriber must currently do.

flatMap/merge

Another area is flatMap and merge which are one of the hottest parts of RxJava. Almost all code of any usefulness is going via merge due to flatMap. It has been mangled significantly to get performance gains. It is worth evaluating whether there are design changes that can facilitate different approaches. I have no obvious things to pursue, but one thought is whether high-performance server systems could benefit from plugging in something like Disruptor to get high performance queueing and dispatch. This could mean supporting a flatMap model that involves a queue draining model whereas today we never "schedule" but always use the producing threads to emit. Should we support an optional model where producer threads just deliver the data and a draining thread emits? We should evaluate the performance benefits for different use cases and environments and considering allowing a flatMap(func, Scheduler) type of override that permits this.

I do think though that the default should be as now where no scheduling occurs and the producer threads always are the ones that emit.

fusion

The lift approach seems to work very efficiently on the JVM and I have not seen evidence of performance gains from function fusion in basic testing I've done, but we should revisit this and determine if we want to implement lift differently.

Today it composes functions. We could change it internally to capture a list of functions that allows analysis and optimization via function fusion if we demonstrate a benefit. This could in fact probably be evaluated in 1.x as well since it shouldn't affect the public API. But I definitely want to revisit the need in RxJava 2.0, particularly since Java 9 as a target may have different performance characteristics.

[akarnokd]

I can't be sure about all implications until I start coding against j.u.c.Flow/reactive-streams specs. A 2.0 branch with JDK 8 target would be nice soon. I have the uncanny feeling implementing the entire library purely on reactive-streams interfaces make things difficult.

[benjchristensen]

The list of issues for 2.0 are marked with the 2.0 milestone: https://github.com/ReactiveX/RxJava/milestones/2.0

Some important ones as we design include:

• 2.0 Design: ReactiveX Design Guidelines and Contract #2781 2.0 Design: ReactiveX Design Guidelines and Contract
• 2.0 Design: Resource Cleanup #2780 2.0 Design: Resource Cleanup
• 2.0 Design: Disposal and Cancellation #2782 2.0 Design: Disposal and Cancellation
• 2.0 Design: Hot and Cold #2785 2.0 Design: Hot and Cold
• 2.0 Design: Test Suite #2783 2.0 Design: Test Suite
• 2.0 Design: Performance Benchmarking #2784 2.0 Design: Performance Benchmarking