On a recent project, working with Elixir, Ecto, and Phoenix, we were encountering difficulty modeling and managing a complex business workflow. This is a fairly typical problem to have on any project—there have been many times I've known that I'm building an implicit finite state machine but don't take the time to make it explicit—but here it was becoming particularly unmanageable. It became clear that we needed to implement a more formal state machine to encode this logic.
Eventually, we implemented a solution that allows us to easily model these workflows in code, in a simple enough way that it also facilitates talking through these workflows and statecharts with non-technical project stakeholders. Read on to learn more about ExState, or try it out to see if it's useful to you.
The Problem
Tracking any complex process or workflow inevitably requires conditional logic
based on the state of entities involved in the workflow. This can easily lead to a
proliferation of fields and states used to track this process. Take an example Order
process, for instance:
defmoduleOrderdouseEcto.Schemaschema"orders"dofield:created_at,:utc_datetimefield:confirmed,:booleanfield:confirmed_at,:utc_datetimefield:shipped,:booleanfield:shipped_at,:utc_datetimefield:shipment_status,:stringfield:cancelled,:booleanfield:cancelled_at,:utc_datetimehas_many:items,ItemendendIn the above schema, we have four fields related to tracking the state of the
order. These fields can change independently and it's unclear how they should
relate to one another. Is %{confirmed: false, shipped: true, cancelled: true} a
valid state? Probably not, but it would require a case by case validation to
enforce that.
The order workflow and state becomes much easier to understand and validate
if we collapse those fields into one state field:
defmoduleOrderdouseEcto.Schemaschema"orders"dofield:state,:stringfield:created_at,:utc_datetimefield:confirmed_at,:utc_datetimefield:shipped_at,:utc_datetimefield:cancelled_at,:utc_datetimehas_many:items,ItemendenddefmoduleOrdersdoaliasEcto.Multidefcancel(order)doiforder.statein["pending","confirmed"]doorder|>Order.changeset(%{status:"cancelled",cancelled_at:DateTime.utc_now()})|>Repo.update()else{:error,"can't cancel this order"}endenddefship(order)doiforder.state=="confirmed"doMulti.new()|>Multi.update(:order,Order.changeset(order,%{status:"shipped",shipped_at:DateTime.utc_now()}))|>Multi.run(:ship,fn_repo,%{order:order}->Shipments.ship(order)end)|>Repo.transaction()else{:error,"order must be in the confirmed state"}endendendWe can then use the states ["created", "confirmed", "shipped", "delivered",
"cancelled"] to represent our order workflow. But we still need to replace the
shipment_state field and to express sub-states of the "shipped" state.
We're also still required to do a lot of explicit checking of the current state
before taking any action.
Statecharts
A lot of the workflows and processes we build in our software can be modeled as finite state machines. The system, or an entity in the system, is in one given state at any time. In that state, the system handles events and transitions to a next state with logic that is dependent upon the current state.
Elixir and Erlang already provide tools for implementing state machine behavior through gen_statem and its predecessor gen_fsm. These are great for implementing stateful processes, but aren't well-suited for state that's stored in a database with a request and response lifecycle.
Our order process resembles a finite state machine, but with a few additional
details. In the "shipped" state, we want to track multiple shipment_status
states. So the "shipped" state itself represents multiple states:
[{"shipped","pending"},{"shipped","in_progress"},{"shipped","arriving_soon"},{"shipped","out_for_delivery"}]Rather than a simple finite state machine, we have a hierarchical state machine, with
certain states like "shipped" representing a lower level of states and
transitions.
ExState
ExState was designed to simplify the definition of states, events, and transitions for simple or hierarchical state machines in Elixir. ExState can also persist these states and metadata to the database through Ecto for use in web apps or other database-backed applications.
Using ExState begins with a definition module. ExState.Definition defines the
workflow macro, which builds a data representation of the state chart and
binds it to the module and the associated functions for transitioning the
workflow with a subject and other context data.
defmoduleOrderWorkflowdouseExState.Definitionworkflow"order"dosubject:order,Orderinitial_state:pendingstate:preparingdoinitial_state:pendingon:cancel,:cancelledstate:pendingdostep:confirmon_completed:confirm,:confirmedendstate:confirmeddostep:shipon_completed:ship,{:<,:shipped}endendstate:shippeddoinitial_state:in_transiton_entry:update_shipped_atstate:in_transitdoon:arriving_soon,:arriving_soonendstate:arriving_soondoon:out_for_delivery,:out_for_deliveryendstate:out_for_deliverydoon:delivered,{:<,:delivered}endendstate:delivereddofinalendstate:cancelleddoon_entry:update_cancelled_atfinalendenddefupdate_shipped_at(%{order:order})update_timestamp(order,:shipped_at)enddefupdate_cancelled_at(%{order:order})update_timestamp(order,:cancelled_at)enddefpupdate_timestamp(order,timestamp)doorder|>Order.changeset(%{timestamp=>DateTime.utc_now()})|>Repo.update()endendThe subject of the workflow is an Ecto model that defines a workflow
association using the has_workflow macro.
defmoduleOrderdouseEcto.SchemauseExState.Ecto.Subjectschema"orders"dofield:created_at,:utc_datetimefield:confirmed_at,:utc_datetimefield:shipped_at,:utc_datetimefield:cancelled_at,:utc_datetimehas_workflowOrderWorkflowhas_many:items,ItemendendThe associated context module can then update the model and its workflow through ExState:
defmoduleOrdersdoaliasEcto.Multi@doc""" Create an order and workflow in a Ecto.Multi transaction."""defcreate_order(attrs)do{:ok,%{order:order,workflow:workflow}}=Multi.new()|>Multi.create(:order,Order.new(attrs))|>ExState.Multi.create(:order)|>Repo.transaction()end@doc""" Load, complete step, and persist."""defconfirm_order(order)doexecution=ExState.load(order){:ok,execution}=ExState.Execution.complete(execution,:confirm){:ok,order}=ExState.persist(execution)end@doc""" Use `ExState.event/3` convenience function to load, transition, and persist."""defcancel_order(order)do{:ok,order}=ExState.event(order,:cancelled)end@doc""" Load, complete, ship, and persist in a transaction."""defship(order)do{:ok,%{ship:shipped_order,shipment:shipment}}=Multi.new()|>ExState.Ecto.Multi.complete(order,:ship)|>Multi.run(:shipment,fn_repo,%{order:order}->Shipments.ship(order)end)|>Repo.transaction()enddefarriving_soon(order)do{:ok,order}=ExState.transition(order,:arriving_soon)endendStates
States are defined in four main forms.
Atomic States
Atomic states have no child states. The following three states are atomic states:
workflow"example"doinitial_state:atomic_astate:atomic_adoon:next,:atomic_bon:done,:atomic_doneendstate:atomic_bdoon:back,:atomic_aendstate:atomic_doneendCompound States
Compound states contain child states and specify an initial state (one of the
child states). The :preparing and :shipped states in the order workflow
are compound states.
Final States
A final state represents a state of either a child state or the entire workflow where the state should be considered "complete."
workflow"example"doinitial_state:astate:adoon_final:bstate:onedoon:did_one,:twoendstate:twodofinalendendstate:bendTransient States
Transient states are used for dynamically resolving the next state based on
conditions defined in the definition module. The transient state immediately
handles a "null" event, :_, and transitions to the first state in the list
that a guard allows.
defmoduleSetupWorkflowdouseExState.Definitionworkflow"setup"doinitial_state:unknownstate:unknowndoon:_,[:accept_terms,:working]endstate:accept_termsstate:workingenddefguard_transition(:unknown,:accept_terms,context)doifcontext.user.has_accepted_terms?do:okelse{:error,:accepted}endendendActions
Actions are useful for triggering side effects on certain events. Actions are
called as functions on the definition module, and should return :error |
{:error, reason} if transactional execution should be halted when an action
cannot be completed. Actions can also return an {:updated, subject} tuple to
replace the updated subject in the execution state.
defmoduleExampleWorkflowsouseExState.Definitionworkflow"example"dostate:workingdoon_entry:send_entry_emailon:cancel,:canceled,actions:[:send_cancelled_email]on_final:done,actions:[:send_final_email]endenddefsend_entry_email(_context),do::okdefsend_final_email(_context),do::okdefsend_cancelled_email(context)do{:updated,Map.put(context,:document,%{document|cancellation_email_sent_at:DateTime.utc_now()})}endendGuards
Guards can ensure that transitions are only made when certain conditions are
met. The guard_transition/3 function on the definition module will be called
during workflow execution with the current state, next state, and the context as
arguments. A guard returns :ok to allow the transition, or {:error, reason
to prevent the transition.
ExState doesn't rescue exceptions in guards or actions, so exception handling behavior is dependent upon the current database transaction context, if any.
Steps
Steps are a convenient way to collapse an implicitly linear set of states and
events into an explicitly ordered list of events. This is useful for exposing
required steps to UI components or API consumers. Steps can also be ignored
through the use_step/2 callback.
state:workingdoinitial_state:adding_nameon_final:reviewingstate:adding_namedoon:name_added,:adding_emailendstate:adding_emaildoon:email_added,:adding_phone_numberendstate:adding_phone_numberdoon:phone_number_added,:confirmingendstate:confirmingdoon:confirmed,:doneendstate:donedofinalendendstate:reviewingThe above workflow could be rewritten using four explicit steps:
defmoduleSetupWorkflowdouseExState.Definitionworkflow"setup"dosubject:account,Accountinitial_state:workingstate:workingdostep:add_namestep:add_emailstep:add_phone_numberstep:confirmon_completed:confirm,:reviewingendstate:reviewingenddefuse_step?(:add_phone_number,context)docontext.account.phone_number_required?enddefuse_step?(_,_),do:trueend{:ok,account}=ExState.create(account){:ok,account}=ExState.complete(account,:add_name){:ok,account}=ExState.complete(account,:add_email){:error,_reason}=ExState.complete(account,:confirm){:ok,account}=ExState.complete(account,:add_phone_number){:ok,%{workflow:%{state:"reviewing"}}}=ExState.complete(account,:confirm)Decisions
Similar to steps, decisions are required events that transition based on the value provided in the decision.
defmoduleReviewWorkflowdouseExState.Definitionworkflow"review"doinitial_state:ratingstate:ratingdostep:rateon_decision:rate,:good,:doneon_decision:rate,:bad,:feedbackendstate:feedbackdostep:provide_feedbackon_completed:provide_feedback,:doneendstate:donedofinalendendend{:ok,review}=ExState.create(review){:ok,review}=ExState.decision(review,:rate,:good)Querying State
You'll likely want to use the workflow state in queries as well. ExState has some builtin queries to help with this:
importExState.Ecto.Queryshipped_orders=Order|>where_any_state(:shipped)|>Repo.all()in_transit_orders=Order|>where_state([:shipped,:in_transit])|>Repo.all()confirmed_orders=Order|>where_step_complete(:confirm)|>Repo.all()Without Ecto
ExState can be used without persisting the workflow to the database, either for testing or in memory use cases. Just use the functions defined on the workflow definition module itself:
%{state:%{name:"delivered"},context:%{order:order}}=execution=OrderWorkflow.new(%{order:order})|>OrderWorkflow.transition_maybe({:completed,:confirm})|>OrderWorkflow.transition_maybe({:completed,:ship})|>OrderWorkflow.transition_maybe(:out_for_delivery)|>OrderWorkflow.transition_maybe(:arriving_soon)|>OrderWorkflow.transition_maybe(:delivered){:error,_reason,execution}=OrderWorkflow.transition(execution,:cancel)Notes
Try ExState on hex, read the docs, or check out the code on GitHub for additional documentation and examples.
Credit to David Khourshid and xstate for excellent documentation, examples, and API inspiration on this topic.