- Support
:on_cancel
inGenStage.from_enumerable/2
- Shuffle the demands on first dispatch for load balancing across consumers
- Allow explicitly specifying
:max_demand
inGenStage.DemandDispatcher
- Do not dispatch when accumulating demand
- Log a clear error messages when stage terminates due to noproc on non-temporary subscription
v1.1 requires Elixir v1.7+.
- Fix warnings on latest Elixir versions
- Support more process specifications in the
:subscribe_to
option - Add callback to allow instrumenting discarded count and function to get estimated buffer size
- Allow events to be discarded in PartitionDispatcher by returning
:none
- Raise for unknown partitionis in PartitionDispatcher
- Improvements to typespecs and error messages
- Add
GenStage.demand/1
- Fix code_change callback implementation
- Fix warnings on Elixir v1.8
This version requires Elixir v1.5+.
- Ensure a
:producer_consumer
stops asking the producer if downstream demand is 0 - Enforce the :hash option for non-int partitions in PartitionDispatcher
Note: this is the last version to support Elixir v1.3 and v1.4.
- Log error on unknown partitions
- Do not allow restart: :permanent in
ConsumerSupervisor
to avoid common pitfalls - Improve and fix types to avoid dialyzer warnings
- Avoid conflict with user specified
@opts
- Mirror
ConsumerSupervisor.init/2
afterSupervisor.init/2
- No longer define default implementations for callbacks, instead declare them as
@optional_callbacks
- Ensure
ConsumerSupervisor
does not send demand when restarting a child
- Support Elixir v1.5 supervisor childspecs in ConsumerSupervisor
- Mark
GenStage.child_spec/1
overridable
- Define Elixir v1.5 childspecs in GenStage and ConsumerSupervisor
- Fix a bug where info messages would be sent out of order in producer consumers
- Fix a bug where handle_cancel would be invoked out of order in producer consumers
- Add
cancel: :transient
to subscription options which does terminate if the exit is:normal
,:shutdown
, or{:shutdown, _}
- Add
GenStage.sync_info/3
andGenStage.async_info/2
which queues an information message to be delivered once the current queue is consumed
- Remove
:max_dynamic
from ConsumerSupervisor - The notification mechanism has been removed from GenStage. For termination, GenStage now uses proper exit signals and
cancel: :transient
has been added as a subscription option.
- Remove the Experimental namespace
- Rename DynamicSupervisor to ConsumerSupervisor
- Move Flow to a separate project: https://github.com/elixir-lang/flow
Except by the module name changes, all APIs remain exactly the same.
- Accumulate demands but don't sum them together. This provides a better ramp up time for producers with multiple consumers
- Add
Flow.group_by/3
andFlow.group_by_key/3
as conveniences aroundFlow.reduce/3
- Add
Flow.map_values/2
for mapping over the values in a key-value based state - Add
Flow.take_sort/3
that efficiently sorts and takes the top N entries
- Ensure BroadcastDispatcher sends demand to itself when custom selector discards events
- Ensure flows started with
Flow.start_link/2
properly terminate if producers terminate - Ensure flows exit locally instead of relying on linked processes exits. With this change,
Flow.run(flow)
andEnum.to_list(flow)
no longer start stages linked directly to the caller but does so through a supervisor
- Add
GenStage.sync_resubscribe/4
andGenStage.async_resubscribe/4
- Improve logs, specs and docs
- Ensure
Flow.departition/4
works onFlow.start_link/1
- Make sure no lingering monitors or messages on the inbox after GenStage.stream/1
- Support a
:selector
option in theBroadcastDispatcher
- Ensure PartitionDispatcher does not create more partitions than necessary
- Pass the events
length
to dispatchers for more performant dispatching
- Introduce count-based windows, process-time windows and session-based windows on Flow
- Support resetting or keeping buffer on Flow watermarks
- Remove
:milliseconds
,:seconds
,:minutes
and:hours
for units in favor of:millisecond
,:second
,:minute
and:hour
. You will get an error if you use the previous values. - Specifying shortcuts to
:hash
has been removed in favor of the:key
option. You will get an error if you use the previous values.
- Ensure uneven partitions emit all windows on
Flow.departition/4
- Properly emit the beginning of the window time on triggers for fixed windows
- Properly count the most recent entry for each fixed window
- Introduce
Flow.departition/5
- Include examples of broadcasters and rate limiters in the documentation
- Allow custom-named, non-integer partitions
- Ensure consumer supervisor respects
min_demand
and does not send demand too soon
- Remove
Flow.new/0
,Flow.new/1
andFlow.new/2
in favor of passing options tofrom_enumerable/2
andfrom_stage/2
- Remove
Flow.partition/3
andFlow.merge/3
in favor of passing the:window
option toFlow.partition/2
andFlow.merge/2
This release moves Flow
from under the GenStage
namespace and into Experimental.Flow
.
- Add
Flow.uniq/2
andFlow.uniq_by/2
- Add
Flow.start_link/2
andFlow.into_stages/3
- Add
Flow.window_join/8
- Unify window and partition APIs
- Support
Flow.Window.global/0
andFlow.Window.fixed/3
- Add
Flow.inner_join/6
- Add
GenStage.demand/2
that allows a producer to accumulate demand as a synchronization mechanism - Improve performance for the partition dispatcher and allow it to change the partitioned event
- Fix a bug where a flow wouldn't terminate if a source stream halts
- Add
Flow.trigger/3
andFlow.trigger_every/4
supporting custom, count and processing-time triggers. Event-time triggers can be implemented viatrigger/3
. Event-time triggers will be added once windows support is included
- Introduce
Flow
with enumerable/stream based operations - Include more information on
:sys.get_status/1
calls for GenStage
- Fix a bug where a
:producer_consumer
stage which filtered events would eventually halt - Fix
format_status/2
results when inspecting GenStage in:observer
- Support notifications
- Introduce
GenStage.stream/1
to stream events as a consumer from a stage - Introduce
GenStage.from_enumerable/2
to start a producer stage that emits events from an enumerable (or a stream)
- Add
GenStage.PartitionDispatcher
- Set default
:max_demand
to 1000 - Use buffer based
:producer_consumer
to avoid escalating demand
- Support
:producer_consumer
type - Support
:infinity
as:buffer_size
(useful for:producer_consumer
)
- Namespace all modules under
Experimental
- Ensure
:cancel
reason does not cascade through the pipeline
- Include GenStage with
:producer
and:consumer
types - Include ConsumerSupervisor implemented as a
GenStage
consumer and that provides the:simple_one_for_one
functionality